Any consumer who is the subject of a consumer score should have the right to see his or her score and to ask for a correction of the score and of the information used in the score.
Author proposed a fair opinion for credit score. However, everyone has its own opinion. Shall we satisfy everyone's correction?
The most salient feature of modern consumer scores is the scores are typically secret in some way. The existence of the score itself, its uses, the underlying factors, data sources, or even the score range may be hidden. Consumer scores with secret factors, secret sources, and secret algorithms can be obnoxious, unaccountable, untrustworthy, and unauditable. Secret scores can be wrong, but no one may be able to find out that they are wrong or what the truth is. Secret scores can hide discrimination, unfairness, and bias. Trade secrets have a place, but secrecy that hides racism, denies due process, undermines privacy rights, or prevents justice does not belong anywhere.
From company perspective, if they disclose their algorithms or factors, it also leads other to copy their strategy. In this sense, I suppose censor from trusted agency is helpful?
Over the past couple of years, Jennifer Oliva, director of the Center for Health and Pharmaceutical Law at Seton Hall University, has set out to examine NarxCare in light of these apprehensions. In a major recent paper called “Dosing Discrimination,” she argues that much of the data NarxCare claims to trace may simply recapitulate inequalities associated with race, class, and gender. Living in a rural area, for example, often requires traveling longer distances for treatment—but that doesn’t automatically signify doctor shopping. Similarly, while it’s a mystery exactly how NarxCare may incorporate criminal justice data into its algorithm, it’s clear that Black people are arrested far more often than whites. That doesn’t mean that prescribing to them is riskier, Oliva says—just that they get targeted more by biased systems. “All of that stuff just reinforces this historical discrimination,” Oliva says.
Due to some historical reasons, what algorithms learns from data also learns prejudice from human. Can scholars develop a method to reduce bias? That's a cliché I suppose, but it seems nothing change for a long time.
Even after Kathryn had read up on NarxCare, however, she was still left with a basic question: Why had she been flagged with such a high score? She wasn’t “doctor shopping.” The only other physician she saw was her psychiatrist.
For this reason, many scholars began to develop interpretable machine learning, focusing on explaining feature importance from each feature. For example, su-in lee develops 'SHAP' for explaining feature contribution for a prediction(score). Thus, people can know which feature leads to high score or low score.
If data pose a risk to groups as much as to individuals, this raises new questions aboutinformed consent. Bernal (2010) argues that since personal data are constantly updated, asystem of real-time ‘collaborative consent’ must be developed. However, this can work onlywhere users are continually connected, literate and aware of the problems of privacy thattheir technology use may pose.
We need to enhance our awareness of privacy protection as well as trying to give notices about what the purpose of using our data to conduct research.
. They then note that a common way to de-anonymise individuals is to mergeand link these network-based identifiers with others gleaned from online data from the area inquestion. However, as they point out, there is little online activity in Coˆte d’Ivoire, and otherdatasets that might help to de-anonymise individuals in this way are not available.
Such issues seriously disturb people's daily life. However it also cultivate a new area of research, 'differential privacy', which tends to release data while protecting people's privacy.
These two documents are unusual for two reasons: first, they show the director of the census shifting the responsibility for the Bureau’s political problems onto one of its division heads and clearly angry with his subordinates; second, they show, for the first time, that the classification by color or by race fell into the political domain.
Serving as political reason, classification by race seems to be a policy that deliberately makes people differ from others.
6 I n t r o d u c t i o n population and not allowing space for an in- between group. By analyzing the procedures, working methods, argumentation, and critiques of the workings of the census, one can document the link that exists between accounting for race and accounting for ethnicity and thus transcend this artificial divide
The classification policy seems to be implemented intentionally to serve some purpose. It leads to bias for different nationality.
For instance, GPT-2’s training data is sourced by scraping out-bound links from Reddit, and Pew Internet Research’s 2016 surveyreveals 67% of Reddit users in the United States are men, and 64%between ages 18 and 29.13 Similarly, recent surveys of Wikipediansfind that only 8.8–15% are women or girls [9].
It seems like that inductive bias exists in the beginning of the research. It turns to be a big problem when we build up a large model.
Forexample, the Perspective API model has been found to associatehigher levels of toxicity with sentences containing identity markersfor marginalized groups or even specific names
It seems like that model overfit to the data so that it classifies a small word as negative markers.
For example, Rohingya refugees have expressed grave concerns that personal data collected by humanitarian organizations may be shared with the government of Myanmar, the same actor that perpetrated atrocities against them (Madianou, 2019). On the corporate side, too often the interests of vulnerable populations are forgotten when their data can be put to other uses, such as to improve facial recognition systems (Madianou, 2019) that may be sold back to governments seeking to keep refugees out.
The data is misused for the bad purpose. It takes algorithm to be unfair in the beginning.
This is true no matter what algorithm is used, as long as it’s designed so that each risk score means the same thing regardless of race.
I think it means calibration for different races...
White defendants get jailed for a risk score of 7, but black defendants get released for the same score. This, once again, doesn’t seem fair. Two people with the same risk score have the same probability of being rearrested, so shouldn’t they receive the same treatment? In the US, using different thresholds for different races may also raise complicated legal issues with the 14th Amendment, the equal protection clause of the Constitution.
Population might be a one of the important factor for interpreting this. Researchers may use percentage of population to determine the threshold but it turns out to be unfair for different races and it confuses the public.
Consider that such data may bedeployed in the service of “algorithmic black-balling”where workers judged unworthy for one position, areultimately sorted into a permanent unemployable caste(Ajunwa, forthcoming 2021
When people try to improve their system via big data, it also strength the bias out of data itself. It inherits people bias in the history and use it as a criteria.
It is powered by haphazard data gathering and spurious correlations, reinforced by institutionalinequities, and polluted by confirmation bias.”9Racial codes are born from the goal of, and facilitate, social control. For instance, in a recent audit ofCalifornia’s gang database, not only do Blacks and Latinxs constitute 87 percent of those listed, but
Sometimes we think we learn from data but it turns out to be a coincidence.
Whether deciding which teacher to hire or fire or which loan applicant to approveor decline, automated systems are alluring because they seem to remove the burden from gatekeepers,who may be too overworked or too biased to make sound judgments.
Automated systems seem to be a double-edged sword. Sometimes it mitigate bias from people, sometimes it induces bias for people.