Appriss is adamant that a NarxCare score is not meant to supplant a doctor’s diagnosis. But physicians ignore these numbers at their peril. Nearly every state now uses Appriss software to manage its prescription drug monitoring programs, and most legally require physicians and pharmacists to consult them when prescribing controlled substances, on penalty of losing their license. In some states, police and federal law enforcement officers can also access this highly sensitive medical information—in many cases without a warrant—to prosecute both doctors and patients.

Over the past two decades, the US Department of Justice has poured hundreds of millions of dollars into developing and maintaining state-level prescription drug databases—electronic registries that track scripts for certain controlled substances in real time, giving authorities a set of eyes onto the pharmaceutical market. Every US state, save one, now has one of these prescription drug monitoring programs, or PDMPs. And the last holdout, Missouri, is just about to join the rest.

While new scores multiply, consumers remain in the dark about many of their consumer scores and about the information included in scores they typically don’t have the rights to see, correct, or opt out of. A primary concern is how these scores affect individuals and meaningful opportunities available to them. Another area of concern is the factors used in new consumer scores, which may include readily commercially available information about race, ethnicity, religion, gender, marital status, and consumer-reported health information.

Predictive scores bring varying benefits and drawbacks. Scores can be correct, or they can be wrong or misleading. Consumer scores – created by either the government or the private sector – threaten privacy, fairness, and due process because scores, particularly opaque scores with unknown ingredients or factors, can too easily evade the rules established to protect consumers.

The issue of unlawfulness over the harm caused by derogatory resultsis a question of considerable debate. For example, in the United States,where free speech protections are afforded to all kinds of speech, includ-ing hate speech and racist or sexist depictions of people and communi-ties, there is a higher standard of proof required to show harm towarddisenfranchised or oppressed people. We need legal protections nowmore than ever, as automated decision-making systems wield greaterpower in society

If there is a technical fix, then what are the constraints that Googleis facing such that eight years later, the issue has yet to be resolved? Asearch for the word “Jew” in 2012 produces a beige box at the bottom ofthe results page from Google linking to its lengthy disclaimer about theresults—which remain a mix of both anti-Semitic and informative sites(see figure 1.13). That Google places the responsibility for bad resultsback on the shoulders of information searchers is a problem, since mostof the results that the public gets on broad or open-ended racial andgendered searches are out of their control and entirely

The internet is full of information borders. Sometimesthose borders follow foreseeable lines, for example accord-ing to geography and language. But there are frequentexceptions. Many information borders are unpredictableand harder to explain. Geographically close countrieswith the same language may have significantly differentresults, and distant countries with unrelated languagesmay have unexpectedly similar results. By automatically

To identify such information regions and borders moreprecisely, we choose to analyze text results, not images,since there are better-defined ways of measuring textsimilarity. We analyze the text using automatic methodsthat reveal which countries have similar and dissimilarresults, enabling us to remap the world by the similarityof the information that a searcher sees.Specifically, we again make searches worldwide usingthe Google-determined default language for the coun-try. Then, given the text results in that language, wemachine-translate the results back into English, againusing Google Translate. Each country’s English resultscan be understood as an approximately hundred-dimensional vector of its most distinctive words, foundvia the tf-idf algorithm [31]. (For example, Japan’s topwords in its results for “god” are “japanese,” “shinto,”“kami” [spirits], and “awe.”) The similarity between twocountries is quantified as the cosine similarity betweentheir vectors. Finally, we use an algorithm called UMAP,which is state-of-the-art for dimensionality reduction[32], to arrange the countries in a two-dimensional spaceand automatically cluster them according to how similartheir search results are.

One of the problems for those trying to see movement clearly using this type of data isnumerical accuracy regarding data subjects. On average, the presence of antennas willcorrelate fairly closely with the presence of people: a mobile provider’s ability to collectlocation data on its users is dependent on their connecting with antennas, and since remotelocations have fewer antennas (de Montjoye et al., 2013), where the population is less dense,fewer signals will be collected. So for example there will be both more signals and morepeople in urban areas than in rural ones. This correspondence between signals andindividuals becomes unreliable, however, when large numbers of people move throughremote areas with few antennas.

First, calling records showwhich of a network’s antennae is being used, and thus the user’s movement from the vicinityof one antenna to another. Alternatively, more specific location data can be gathered as aphone’s SIM card automatically checks in with its nearest antenna.

Studies on the construction of race in the United States often mention census categories, either as an indication of the stages of its evolution or to emphasize the federal government’s participation in the process. But the racial categories of the census are most often perceived as a sign of racism at work in society, rather than the subject of a more in- depth investigation.

The Census Bureau had hoped to make use of Mexicans’ feelings of ethnic pride to convince them to participate enthusiastically in the 1930 census.

In spite of this trend toward greater complexity, the system of racial catego-ries, so important in American public statistics, still relied on the primordial opposition between whites and blacks. In fact, the relationship by this time had been solidified by locking the definition of the black population, so that it was finally stable, while the definition and limits of the “white race” were called into question by the importance of the immigration issue, which was also a racial issue even if not a question of color, and by the creation, with the 1930 census, of a Mexican race.

“They are not only failing to prioritize hiring more people from minority communities, they are quashing their voices,” she said.

undar Pichai, chief executive of Alphabet, Google’s parent company, has compared the advent of artificial intelligence to that of electricity or fire, and has said that it is essential to the future of the company and computing

All of these approaches take time and are most valuable whenapplied early in the development process as part of a conceptual in-vestigation of values and harms rather than as a post-hoc discoveryof risks [72]. These conceptual investigations should come beforeresearchers become deeply committed to their ideas and thereforeless likely to change course when confronted with evidence of pos-sible harms. This brings us again to the idea we began this sectionwith: that research and development of language technology, atonce concerned with deeply human data (language) and creatingsystems which humans interact with in immediate and vivid ways,should be done with forethought and care

Speed is key. There are technologies, such as semi-supervised learning which requires very little labeled data, that could eventually allow tech companies to develop language services without seeking out cultural knowledge, Mehelona says.

Social media platforms like Facebook are ill-equipped to combat these trends as they simply have not invested the resources in personnel and AI systems that understand local languages and social tensions at play. Their business models raise questions as to whether this trend will be reversed without a major paradigm shift.

There is no algorithm that can fix this; this isn’t even an algorithmic problem, really. Human judges are currently making the same sorts of forced trade-offs—and have done so throughout history.

We gave you two definitions of fairness: keep the error rates comparable between groups, and treat people with the same risk scores in the same way. Both of these definitions are totally defensible! But satisfying both at the same time is impossible. 

We need public buy-in (quite literally) for the idea that successful, equitable automation means a sociotechnical system in which workers play a central role, whether through directly or indirectly working with machines, and are compensated accordingly. 

First, the impenetrable black box of automatedhiring casts doubts on the notion of equal opportunityfor all workers, as the fairness of criteria used in hiringcannot be verified. A nontransparent hiring process alsocreates more difficulty for discovering and redressingdisparities in hiring. Second, the opaque nature of howdata collected from workers could be evaluated ordeployed both by present and future employers presentschallenges to worker personhood, worker autonomy,and social mobility.

This filtering has been used to show higher-paying job adsto men more often than to women, to charge more for standardized test prep courses to people in areaswith a high density of Asian residents, and many other forms of coded inequity.

Despite decades of scholarship on the social fabrication of group identity,tech developers, like their marketing counterparts, are encoding race, ethnicity, and gender asimmutable characteristics that can be measured, bought, and sold

use of the company’s AI for military applications.

The ubu-Ntu framework for understanding the ethical implications and human rights risks of automated decision-making systems (ADMS) relies on examining the nature of the interconnected and layered relationships relating to the creation and use of these systems.

eo-colonialism, like colonialism, is an attempt to export the social conflicts of the capitalist countries. The temporary success of this policy can be seen in the ever widening gap between the richer and the poorer nations of the world. But the internal contradictions and conflicts of neo-colonialism make it certain that it cannot endure as a permanent world policy.

Large data companies have no responsi-bility to make their data available, and they have total control over who getsto see them.

Both behavioral and articulated networks have great value to researchers,but they are not equivalent to personal networks. For example, although con-tested, the concept of ‘tie strength’ is understood to indicate the importanceof individual relationships (Granovetter 1973). When mobile phone datasuggest that workers spend more time with colleagues than their spouse, thisdoes not necessarily imply that colleagues are more important than spouses.Measuring tie strength through frequency or public articulation is a commonmistake: tie strength – and many of the theories built around it – is a subtlereckoning in how people understand and value their relationships with otherpeople. Not every connection is equivalent to every other connection, andneither does frequency of contact indicate strength of relationship. Further,the absence of a connection does not necessarily indicate that a relationshipshould be made