Since I closed down the Dish, my bloggy website, five years ago, after 15 years of daily blogging, I have not missed the insane work hours that all but broke my health.

Anditisclearthatthislackofsocialconsciousnessisinfactadistincteconomiclossinaveryconcretesense,aswellofcourseasalossinthepossiblewell-runningofapoliticalsystem.IapproachthisfromthepointofviewofaneconomistsoIspeakofthefail-uresofthepricesystem;Iamsureonecouldcometothesameendfromotherpointsofview.Butstartingfromthispointofview,thefactthatwecannotmediateallourre-

Thereisstillanothersetofinstitutions,ifthatistherightword,Iwanttocalltoyourattentionandmakemuchof.Theseareinvisibleinstitutions:theprinciplesofethicsandmorality.Certainlyonewayoflookingatethicsandmorality,awaythatiscompatiblewiththisattemptatrationalanaly-sis,isthattheseprinciplesareagreements,consciousor,inmanycases,unconscious,tosupplymutualbenefits.Theagreementtotrusteachothercannotbebought,asIhavesaid;itisnotevennecessarilyveryeasyforittobeachievedbyasignedcontractsayingthatwewillworkwitheachother.

Outcomes
   ---
Technology
   ---
Institutions
    ---
Culture
     ---
Pyscho-Ontology

Societiesintheirevolutionhavedevelopedimplicitagree-mentstocertainkindsofregardforothers,agreementswhichareessentialtothesurvivalofthesocietyoratleastcontributegreatlytotheefficiencyofitsworking.Ithasbeenobserved,forexample,thatamongthepropertiesofmanysocietieswhoseeconomicdevelopmentisbackwardisalackofmutualtrust.

Therearemanyotherorganizationsbesidethegovern-mentandthefirm.Butallofthem,whetherpoliticalpartyorrevolutionarymovement,universityorchurch,sharethecom-moncharacteristicsoftheneedforcollectiveactionandtheallocationofresourcesthroughnonmarketmethods.

Butinternally,andespeciallyatlowerlevels,therelationsamongtheemployeesofafirmareverydifferentfromthearm’slengthbargainingofourtextbooks.AsHer-bertSimonhasobserved,anemploymentcontractisdifferentinmanywaysfromanordinarycommoditycontract;anemployeeissellingwillingnesstoobeyauthority,aconceptofcentralimportancetowhichIwillreturninalaterchapter.

Iamnotinterestedheresomuchinthesespecificexamplesastoshowthatsomethinglikethisoccursinmoresubtlecontexts.Considerwhatisthoughtofasahigherormoreelusivevaluethanpollutionorroads:trustamongpeople.Nowtrusthasaveryimportantpragmaticvalue,ifnothingelse.Trustisanimportantlubricantofasocialsystem.Itisextremelyefficient;itsavesalotoftroubletohaveafairdegreeofrelianceonotherpeople’sword.Unfortunatelythisisnotacommoditywhichcanbeboughtveryeasily.Ifyouhavetobuyit,youalreadyhavesomedoubtsaboutwhatyou’vebought.Trustandsimilarvalues,loyaltyortruth-telling,areexamplesofwhattheeconomistwouldcall“ex-ternalities.”Theyaregoods,theyarecommodities;theyhavereal,practical,economicvalue;theyincreasetheefficiencyofthesystem,enableyoutoproducemoregoodsormoreofwhatevervaluesyouholdinhighesteem.Buttheyarenotcommoditiesforwhichtradeontheopenmarketistech-nicallypossibleorevenmeaningful.

Rationality,afterall,hastodowithmeansandendsandtheirrelation.Itdoesnotspecifywhattheendsare.Itonlytriestomakeusawareofthecongruenceordissonancebetweenthetwo.Soultimatelyanyvaluediscussionmustcometoaresttemporarilyonunanalyzedpostulates.

AllItrytoinsisthereisthatsomesenseofrationalbalancingofendsandmeansmustbeunderstoodtoplayamajorroleinourunderstandingofourselvesandoursocialrole.Letmeil-lustratebypresentingor,moreprecisely,caricaturingsomethoughttendencies.Wehaveone,looselycalled“thenewLeftthought,”notsonewperhaps;someofuswhohavereadalittlebitofthehistoryofthoughthaveheardofanarcho-syndicalismbefore.BakuninandSorelhadspokentothesamepointmanyyearsago.Butitisarealone.Thereisademandforwhatmightbetermedsincerity,foracompleteunity

TheNewRight,initslibertarianrepresentatives,alsoresolvestheconflictinitsownway.Itseekstodeny,oratleastminimize,theroleofthestateandofcollectiveactionandresponsibility,andsubstitutesfortheseclaims,withtheirdifficultmoralandpowerconsequences,theworshipofthemarket.

to explain the world of interactions and outcomes occurring at multiple levels, we also have to be willing to deal with complexity instead of rejecting it. some mathematical models are very useful for explaining outcomes in particular settings. We should continue to use simple models where they capture enough of the core underlying structure and incentives that they usefully predict outcomes. When the world we are trying to explain and improve, however, is not well described by a simple model, we must continue to improve our frameworks and theories so as to be able to understand complexity and not simply reject it.

B. The Central Role of Trust in Coping with Dilemmaseven though arrow (1974) long ago pointed to the crucial role of trust among participants as the most efficient mechanism to enhance transactional outcomes, collective-action theory has paid more attention to payoff functions than to how individuals build trust where others are recip-

simply assuming that humans adopt norms, however, is not sufficient to predict behavior in a social dilemma, especially in very large groups with no arrangements for communication. even with strong preferences to follow norms, “observed behavior may vary by context because the perception of the ‘right thing’ would change” (de oliveira, croson, and eckel 2009: 19). various aspects of the context in which individuals interact affect how indi-viduals learn about the situation they are in and about the others with whom they are interacting. individual differences do make a difference, but the context of interactions also affects behavior over time (Walker and ostrom 2009). Biologists recognize that an organism’s appearance and behavior are affected by the environment in which it develops.for example, some plants produce large, thin leaves (which enhance photosynthetic photon harvest) in low light, and narrow, thicker leaves (which conserve water) in high light; certain insects develop wings only if they live in crowded conditions (and hence are likely to run out of adequate food in their current location). such environmentally contingent development is so commonplace that it can be regarded as a universal property of living things. (Pfennig and ledón-rettig 2009: 268)social scientists also need to recognize that individual behavior is strongly affected by the context in which interactions take place rather than being simply a result of individual differences.

428their role in climate change-related emissions and carbon sequestration (canadell and raupach 2008), the biodiversity they contain, and their con-tribution to rural livelihoods in developing countries. a “favorite” policy rec-ommendation for protecting forests and biodiversity is government-owned protected areas (terborgh 1999). in an effort to examine whether govern-ment ownership of protected areas is a necessary condition for improving forest density, hayes (2006) used ifri data to compare the rating of forest density (on a five-point scale) assigned to a forest by the forester or ecologist who had supervised the forest mensuration of trees, shrubs, and ground-cover in a random sample of forest plots.9of the 163 forests included in the analysis, 76 were government-owned forestslegally designated asprotected forestsand 87 were public, private, or communally owned forested lands used for a diversity of purposes. no statistical difference existed between the forest density in officially designated protected areas versus other forested areas. Gibson, Williams, and ostrom (2005) examined the monitoring behav-ior of 178 forest user groups and found a strong correlation between the level of monitoring and a forester’s assessment of forest density even when controlling for whether users were formally organized, whether the users were heavily dependent on a forest, and the level of social capital within a group.chhatre and agrawal (2008) have now examined the changes in the condition of 152 forests under diverse governance arrangements as affected by the size of the forest, collective action around forests related to improve-ment activities, size of the user group, and the dependence of local users on a forest. they found that “forests with a higher probability of regeneration are likely to be small to medium in size with low levels of subsistence depen-dence, low commercial value, high levels of local enforcement, and strong collective action for improving the quality of the forest” (ibid.: 1327). in a second major analysis, chhatre and agrawal (2009) focus on factors that affect tradeoffs and synergies between the level of carbon storage in forests and their contributions to livelihoods. they find that larger forests are more effective in enhancing both carbon and livelihoods outcomes, particularly when local communities also have high levels of rule-making autonomy.

since the design principles are described extensively in e. ostrom (1990, 2005), i will list only a brief updated list as developed by cox, arnold, and villamayor-tomás (2009):1a.User Boundaries: clear and locally understood boundaries between legitimate users and nonusers are present.1B. resource Boundaries: clear boundaries that separate a specific common- pool resource from a larger social-ecological system are present.2a. congruence with local conditions: appropriation and provision rules are congruent with local social and environmental conditions.2B. appropriation and Provision: appropriation rules are congruent with provision rules; the distribution of costs is proportional to the distribution of benefits.3. collective-choice arrangements: Most individuals affected by a resource regime are authorized to participate in making and modifying its rules.4a. Monitoring Users: individuals who are accountable to or are the users monitor the appropriation and provision levels of the users.4B. Monitoring the resource: individuals who are accountable to or are the users monitor the condition of the resource.5.Graduated sanctions: sanctions for rule violations start very low but become stronger if a user repeatedly violates a rule.6. conflict-resolution Mechanisms: rapid, low-cost, local arenas exist for resolving conflicts among users or with officials.7.Minimal recognition of rights: the rights of local users to make their own rules are recognized by the government.8. nested enterprises: When a common-pool resource is closely connected to a larger social-ecological system, governance activities are organized in multiple nested layers.the design principles appear to synthesize core factors that affect the prob-ability of long-term survival of an institution developed by the users of a re-source. cox, arnold, and villamayor-tomás (2009) analyzed over 100 studies by scholars who assessed the relevance of the principles as an explanation of the success or failure of diverse common-pool resources.

schlager and ostrom (1992) drew on the earlier work of John r. commons ([1924] 1968) to conceptualize property-rights systems as containing bundles of rights rather than a single right. the meta-analysis of existing field cases helped to identify five property rights that individuals using a common-pool resource might cumulatively have: (1) access – the right to enter a specified property,4 (2) Withdrawal – the right to harvest specific products from a re-source, (3) Management – the right to transform the resource and regulate

in cPr dilemmas where individuals do not know one another, cannot communicate effectively, and thus cannot develop agreements, norms, and sanctions, aggregate predictions derived from models of rational individuals in a noncooperative game receive substantial support. these are sparse environments and full rationality appears to be a reasonable assumption in them. (e. ostrom, Gardner, and Walker 1994: 319)

this was an immense effort. More than two years was devoted to develop-ing the final coding manual (e. ostrom et al. 1989). a key problem was the minimal overlap of variables identified by case study authors from diverse dis-ciplines. the team had to read and screen over 500 case studies in order to identify a small set of cases that recorded information about the actors, their strategies, the condition of the resource, and the rules-in-use.3a common set of variables was recorded for 44 subgroups of fishers who harvested from inshore fisheries (schlager 1990, 1994) and 47 irrigation systems that were managed either by farmers or by a government (tang 1992, 1994).of the 47 irrigation systems included in the analysis, 12 were managed by governmental agencies of which only 40 percent (n = 7) had high perfor-mance. of the 25 farmer-managed, over 70 percent (n = 18) had high perfor-mance (tang 1994: 234). rule conformance was a key variable affecting the adequacy of water over time (ibid.: 229). none of the inshore fishery groups analyzed by schlager were government-managed and 11 (25 percent) were not organized in any way. the other 33 subgroups had a diversity of informal rules to define who was allowed to fish in a particular location and how har-vesting was restricted (schlager 1994: 260)

the national research council (nrc) established a commit-tee in the mid-1980s to assess diverse institutional arrangements for effective conservation and utilization of jointly managed resources.

netting 1972

4. are rational individUals helPlessly traPPed in social dileMMas?the classic assumptions about rational individuals facing a dichotomy of or-ganizational forms and of goods hide the potentially productive efforts of in-dividuals and groups to organize and solve social dilemmas such as the over-harvesting of common-pool resources and the underprovision of local public goods. the classic models have been used to view those who are involved in a Prisoner’s dilemma game or other social dilemmas as always trapped in the situation without capabilities to change the structure themselves. this ana-lytical step was a retrogressive step in the theories used to analyze the human condition. Whether or not the individuals who are in a situation have capaci-ties to transform the external variables affecting their own situation varies dramatically from one situation to the next. it is an empirical condition that varies from situation to situation rather than a logical universality. Public in-vestigators purposely keep prisoners separated so they cannot communicate. the users of a common-pool resource are not so limited

in the 1970s, the earlier work on effects of diverse ways of organizing the provision of water in metropolitan areas was extended to policing and public safety. We found that while many police departments served 80 metropolitan areas that we studied, duplication of services by more than one department to the same set of citizens rarely occurred (e. ostrom, Parks, and Whitaker 1978). further, the widely held belief that a multiplicity of departments in a metropolitan area was less efficient was not found. in fact, the “most efficient producers supply more output for given inputs in high multiplicity metropolitan areas than do the efficient producers in metropolitan areas with fewer producers” (e. ostrom and Parks 1999: 287). Metropolitan areas with large numbers of autonomous direct service producers achieved higher

‘Polycentric’ connotes many centers of decision making that are formally independent of each other. Whether they actually function independently, or instead constitute an interdependent system of relations, is an empirical question in particular cases. to the extent that they take each other into account in competitive relationships, enter into various contractual and cooperative undertakings or have recourse to central mechanisms to resolve conflicts, the various political jurisdictions in a metropolitan area may function in a coherent manner with consistent and predictable patterns of interacting behavior. to the extent that this is so, they may be said to function as a ‘system’. (v. ostrom, tiebout, and Warren 1961: 831–32)drawing on the concept of a public service industry (Bain 1959; caves 1964; v. ostrom and e. ostrom 1965), several studies of water industry performance were carried out in diverse regions of california during the 1960s (v. ostrom 1962; Weschler 1968; Warren 1966; e. ostrom 1965). substantial evidence was found that multiple public and private agencies had searched out productive ways of organizing water resources at multiple scales contrary to the view that the presence of multiple governmental units without a clear hierarchy was chaotic. further, evidence pointed out three mechanisms that increase productivity in polycentric metropolitan areas: (1) small- to medium-sized cities are more effective than large cities in monitor-ing performance of their citizens and relevant costs, (2) citizens who are dissatisfied with service provision can “vote with their feet” and move to jurisdictions that come closer to their preferred mix and costs of public services, and (3) local incorporated communities can contract with larger producers and change contracts if not satisfied with the services provided, while neighborhoods inside a large city have no voice.

the market was seen as the optimal institution for the production and exchange of private goods. for nonprivate goods, on the other hand, one needed “the” government to impose rules and taxes to force self-interested individuals to contribute necessary resources and refrain from self-seeking activities. Without a hierarchical government to induce compliance, self-seeking citizens and officials would fail to generate efficient levels of public goods, such as peace and security, at multiple scales (hobbes [1651] 1960; W. Wilson 1885). a single governmental unit, for example, was strongly recommended to reduce the “chaotic” structure of metropolitan governance, increase efficiency, limit conflict among governmental units, and best serve a homogeneous view of the public (anderson and Weidner 1950; Gulick 1957; friesema 1966). this dichotomous view of the world explained patterns of interaction and outcomes related to markets for the production and exchangeof strictly private goods (alchian 1950), but it has not adequately accounted for internal dynamics within private firms (Williamson 1975, 1986). nor does it adequately deal with the wide diversity of institutional arrangements that

Gregor Aisch, Adam Pearce, and Karen Yourish, “The Divide Between Red and Blue America Grew Even Deeper in 2016,” The New York Times, November 10, 2016; See also Gregor Aisch, Adam Pearce, and Karen Yourish, “How Large Is the Divide Between Red and Blue America?,” The New York Times, November 4, 2016; See also David Wasserman, “Purple America Has All But Disappeared,” FiveThirtyEight, March 8, 2017.

What Divides The Parties Now?The parties are divided on both social/identity and economic issues, but more so on identity issues. The gaps between the Clinton and Trump voters on questions of racial resentment, immigration, attitudes toward Muslims, and moral issues are consistently wide. There is very little overlap between the two camps on these issues.By contrast, although the parties are divided on economic issues, there is more overlap. Particularly in the Republican Party, there are a wide range of views on economic issues, now that the party has expanded to include more and more populists who were formerly Democrats.

Though many on the far left argue that Clinton would have won had she been more progressive and excited more Sanders voters, the data here suggest that Clinton may have lost some Democratic voters because her campaign was too left leaning, particularly on the identity and social issues, but perhaps also some issues of government intervention as well.

In Global Catastrophic Risks 2016, we referred to a number used in the Stern Review on the Economics of Climate Change: a 0.1% annual chance of human extinction. Stern uses this as a modelling assumption for discussing discount rates. There is a small amount of discussion of this figure in the Stern Review. It is clear that Stern did not intend the figure as an estimate. We’ve had a critique of our use of the figure forwarded to us, and we think its analysis is useful. We had no intention of using this figure in a misleading way, and we agree that we made a mistake in how we presented this figure. We should have been clearer about what the status of the number in the Stern Review was and about how we intended to use the comparison. Throughout the rest of the report, we are very explicit that we do not believe it is possible to make robust probability estimates of extinction or catastrophic risk and do not attempt to (except for asteroid and super-volcano risk). This mistake does not affect the validity of the main points of the report – that global catastrophic risks are worth addressing and that there are things we can do to address them. In our report, we originally wrote that: “It is easy to be misled by the apparently low probabilities of catastrophic events. The UK’s Stern Review on the Economics of Climate Change suggested a 0.1% chance of human extinction each year, similar to some rough estimates of accidental nuclear warfare. At first glance, this may seem like an acceptable level of risk. Moreover, small annual probabilities compound significantly over the long term. The annual chance of dying in a car accident in the United States is 1 in 9,395. However, this translates into an uncomfortably high lifetime risk of 1 in 120. Using the annual 0.1% figure from the Stern Review would imply a 9.5% chance of human extinction within the next hundred years.” We were aware that the Stern Review used this figure merely as a modelling assumption, and were trying to give a concise accurate statement. Our intention in using the figure from the Stern Review was not to try to pin down an accurate estimate of the likelihood of global catastrophe, but to demonstrate that existing serious analysis treats the 0.1% probability as a plausible modeling assumption, which would have consequences that are interesting and non-intuitive. We also had a full-page summary pull-quote, which said: “The UK’s Stern Review on the Economics of Climate Change suggested a 0.1% chance of human extinction each year. If this estimate is correct, a typical person is more than five times as likely to die in an extinction event as in a car crash.” This implies more confidence in the 0.1% figure than either we felt or expect the Stern Review to have felt, and more than our argument required. The car crash comparison was picked up in The Atlantic, which reported it as an unconditional claim and emphasised it in their article. We did not intend to argue that the 0.1% figure was an accurate estimate of extinction risk (as we did not plan to offer an estimate of extinction risk), so this was inadvertently misleading to Atlantic readers. We believe that in general The Atlantic stood out by doing an excellent job of engaging constructively with our work. We are also sorry in particular that we allowed the word ‘estimate’ to enter the soundbite on the full page. This error occurred at a late stage in the editing; the word was introduced to avoid an ambiguity, but not subjected to proper review. We have carefully reviewed our language concerning the Stern Review, and written to our partners at the Global Challenges Foundation who published the report to change this to: “The probabilities of these catastrophic events are low but not negligible. Moreover, small annual probabilities compound significantly over the long term. We do not know of a robust estimate of the annual probability of global catastrophic risk. Nor do we believe that we are able to create a robust estimate because the uncertainties in key parameters are so large. However, for extinction risks some experts have suggested that a 0.1% annual chance of extinction is within the range of plausible orders of magnitude. A 2008 Oxford survey of expert judgement on the topic implied an average annual extinction risk over the next century of around 0.2%. [1] The UK’s Stern Review on the Economics of Climate Change used 0.1% as an upper bound modeling assumption for annual extinction risk. Now let’s suppose that the chance of extinction were 0.1% per year and consider the consequences. It may seem at first glance that this would be an acceptable level of risk. However, that would mean an individual would be more than five times as likely to die in an extinction event than a car crash. Moreover, these small annual probabilities add up, so that the chance of extinction within the next century under this scenario is 9.5%.  A global catastrophe, which involves the death of 10% of the global population, is more likely than an event that involves human extinction. As a result, even if 0.1% were on the high side for extinction risk, it might be of the appropriate order of magnitude for global catastrophic risk.” We are also correcting a citation and adding a citation to [1] Sandberg, A. & Bostrom, N. (2008): “Global Catastrophic Risks Survey”, Technical. Report #2008-1, Future of Humanity Institute, Oxford University: pp. 1-5. We are also making the corresponding changes in the one-page soundbite and will also write to The Atlantic to inform them of the inadvertent inaccuracy in the article, and offer to help in correcting the nuance of the article.

(J) removing greenhouse gases from the 19atmosphere and reducing pollution by restoring 20natural ecosystems through proven low-tech so-21lutions that increase soil carbon storage, such 22as land preservation and afforestation;

(C) meeting 100 percent of the power de-18mand in the United States through clean, re-19newable, and zero-emission energy sources, in-20cluding—

2) the goals described in subparagraphs (A) 18through (E) of paragraph (1) (referred to in this 19resolution as the ‘‘Green New Deal goals’’) should 20be accomplished through a 10-year national mobili-21zation (referred to in this resolution as the ‘‘Green 22New Deal mobilization’’) that will require the fol-23lowing goals and projects—

So far this year, regulatory credits account for $1.18 billion, or 7% of total automotive revenue.

Revenue rose to a record $8.77 billion from $6.30 billion a year earlier. Analysts had expected revenue of $8.36 billion, according to IBES data from Refinitiv.Excluding items, Tesla posted a profit of 76 cents per share. It reported net income of $331 million, or $874 million excluding stock-based compensation awards given to Musk.Revenue from the sale of regulatory credits made up $397 million. Without that revenue, Tesla would not have achieved a profitable quarter.

For the past four years, I’ve followed a group of steelworkers in Indiana — men and women, Black and white — who had worked at a factory that moved to Mexico. I watched them agonize about whether to train their Mexican replacements, or stand with their union and refuse. I watched them grieve the plant like a parent. I followed them as they applied for new jobs, some of which paid half as much as they made before.A machinist named Tim carried his steelworker union card in his wallet for years after the factory closed, just to remind himself who he was. Tim grew up in a union household. His dad had been an autoworker; his grandfather, a coal miner.“We always voted Democrat because they looked after the little man,” Tim told me. “My father went to his grave and I can guarantee you he never voted for a Republican.”Tim had such faith in Democrats that he didn’t worry when President Bill Clinton pushed the North American Free Trade Agreement over the finish line in 1993. Nor did he worry when Mr. Clinton normalized trade with China in 2000. But then the factory where Tim worked moved to Shanghai. And the next one moved to Mexico.Editors’ PicksA Korean Store Owner. A Black Employee. A Tense Neighborhood.The Problem of Free Speech in an Age of Disinformation5 Things to Avoid on Prime DayAdvertisementContinue reading the main storyBy the time I met Tim, he loathed the Clintons and the Democratic Party. Democrats had gotten in bed with the corporations, while no one was looking. Tim felt betrayed, and politically abandoned — until Mr. Trump came along.

It must be able to interface with the current economic system; must be able to move resources from the current system into the transitional system.Must lead to a new attractive basin that moves a critical mass of resources to the new system, that past a tipping point becomes auto-poetic. (This probably requires out-competing the current system, in a way that can scale to everyone, while obsoleting the destructive forms of competition within the new system – the last act of win/lose gaming dynamics as they transcend themselves.)

It must disincentive all activity that could lead towards catastrophic threats, intentionally or unintentionally; must not incent local or near term positives that increase probability of global or long term negatives. This is particularly critical to get right regarding the incentives on the development of exponential technologies.

They are currently all fundamentally inadequate to the scope, urgency, and nature of the issues and dynamics civilization now faces and must undergo a discrete phase shift to axiomatically restructured, higher order systems.

Economics is a facet of social architecture, inseparable from infrastructure, culture, governance, law, defense, information systems, education and human development, etc. These systems co-evolve and co-influence each other.

The complexity science and technological capacity for closed loop regenerative technology ecosystems

The sensor and data science capacity to maintain a real-time balance sheet of the global commons

Computer science innovations (distributed ledgers, public key cryptography, div/merge, matching algorithms, etc) offering solutions to critical issues in game theory (byzantine generals, etc), intellectual property (partial attribution, etc), and information fidelity and access (high signal to noise ratio distributed sense-making), etc.

For all systems of structural incentive, the incentive of any actor (individual or group), must be rigorously aligned with the well-being of all other agents in the system and of the commons writ large. Ie, all externalities must be internalized – all the consequences of activity within the system must be included in the system’s accounting.

Kennedy’s performance was shockingly sub-par and created a clear impression that he had no particular reason, other than personal ambition, for seeking the highest office in the land. “He was running because he wanted to be president,” Carter’s chief of staff later noted. “That was not such an unusual motive, but most aspirants figure out some way to disguise it better.”

Historians and political scientists see the matter differently today. Kennedy’s own vote counters later conceded that he lost 59 out of 70 white precincts in Gary. While Kennedy’s internal polls showed him faring better than might be expected among former supporters of George Wallace’s bid for the Democratic nomination four years earlier, he nevertheless struggled to retain working-class, white ethnic voters and relied instead on robust turnout in minority neighborhoods for his electoral cushion.

Fifth, and most challenging, we can work to reverse the divergence between the centre and the periphery. The previous four elements would help with this. But greater policy efforts are needed to give regions, where possible, a critical mass of knowledge jobs so they can connect with the leading economic activity in national centres.

These changes are not, on the whole, the fault of globalisation, that scapegoat of the populist insurgency, but of technology-driven changes combined with policies that have reinforced the underlying forces of divergence.

Agreement

Life Itself currently has two hubs: Berlin and Bergerac.

weller, wiser

Finally, we believe in Getting Things Done (opens new window). #

it is important to understand some of the key tools and processes that we use

s

“stuff”

Theoretic chaos has replaced the idealistic thinking of old--and, unable to reconstitute theoretic order, men have condemned idealism itself. Doubt has replaced hopefulness--and men act out a defeatism that is labeled realistic. The decline of utopia and hope is in fact one of the defining features of social life today. The reasons are various: the dreams of the older left were perverted by Stalinism and never re-created; the congressional stalemate makes men narrow their view of the possible; the specialization of human activity leaves little room for sweeping thought; the horrors of the twentieth century symbolized in the gas ovens and concentration camps and atom bombs, have blasted hopefulness. To be idealistic is to be considered apocalyptic, deluded. To have no serious aspirations, on the contrary, is to be "tough-minded."

It has been said that our liberal and socialist predecessors were plagued by vision without program, while our own generation is plagued by program without vision. All around us there is astute grasp of method, technique--the committee, the ad hoc group, the lobbyist, the hard and soft sell, the make, the projected image--but, if pressed critically, such expertise in incompetent to explain its implicit ideals. It is highly fashionable to identify oneself by old categories, or by naming a respected political figure, or by explaining "how we would vote" on various issues.

No Utopias

They also noted that the OCEAN contract “contained a pause function for unforeseen emergencies like this one,” which calls into question the claims made by the developers of these protocols who say their platforms are “decentralized.” If they were truly decentralized, then it would not be possible to shut them down, even if there is an emergency.

Marcuse’s concept of essence is not transcendental but historical. That is, there is no human essence apart from historical context.

In a nutshell, what Marcuse sees in the 1844 Manuscripts is an analysis of the social conditions for a communist revolution. The revolution itself requires the development of radical subjectivity. Radical subjectivity refers to the development of a form of self-consciousness that finds present social and economic conditions intolerable. The radical act is a refusal of these conditions and an orientation toward social transformation. To make sense of Marcuse’s position we must ask: what are these intolerable conditions and how are they produced? Philosophical anthropology and radical subjectivity are connected here insofar as the intolerable conditions that must be overcome by revolution or the radical act represent social distortions of the human essence. It is Marx and Hegel who provided Marcuse with a philosophical anthropology that discloses human essence and the social mechanisms by which it is distorted. The key category here is that of “alienation” which cannot be understood without examining the role of labor and objectification.

In this context, “anthropology” does not refer to the study of past cultures, as it often does in the United States. Instead, it refers to the German idea of anthropology which is more of a philosophical and social scientific examination of human nature.

At the global level none of the 20 targets have been fully achieved, though six targets have been partially achieved (Targets 9, 11, 16, 17, 19 and 20). Examining the 60 specific elements of the Aichi Biodiversity Targets, seven have been achieved and 38 show progress. Thirteen elements show no progress or indicate a move away from the target, and for two elements the level of progress is unknown. The table on the following pages provides an overview of the progress made towards each of the 20 Aichi Biodiversity Targets.

But my story line would be different from Jack’s: it would document not the exhaustion of utopian energies, but rather a continual, dedicated struggle for their renewal. Habermas would be the hero of the book, naturally. His contributions to the “unfinished project” of modernity would provide the dominant narrative arc. In turning away from Heidegger, in channeling the legacy of the Frankfurt School in new directions, in never shying away from public debate and exchange, Habermas has opposed valiantly the forces of what he once called “the new obscurity” — a term that seems to describe only too perfectly our current situation, marked as it is by the forces of social media disinformation, political polarization, and xenophobia. If only in a modest sense, Habermas has kept utopia alive. [28] Invoking the work of John Rawls, he has gone so far as to defend “the realistic utopia of human rights.” [29] It is something, even if it is not, as some scholars have suggested, everything. [30]

One of the books for which Jack Diggins was most well known was Up from Communism, a group portrait of four American intellectuals — Max Eastman, John Dos Passos, James Burnham, and Will Herberg — who over time traded their youthful flirtations with leftist radicalism for supposedly more sober forms of conservatism. First published in 1975, the book was, in its own way, an account of the “exhaustion of utopian energies.” It was about thinkers for whom “the problem with utopia” became “not how to realize it but how to prevent its realization.” [25] In other words, up from communism meant down with utopia.

You can read more about Life Itself’s offerings on our website:

GitHub/ GitLab Permissions

please email the OPS team (office@datopian.com) with the following information:

Piketty’s account of the past 40 years is less a story of capital being unleashed (as most histories of neoliberalism have it) than of progressive ideologies running out of steam. The failure of communism played a crucial role in this, producing a new fatalism about the capacity of politics to deliver equality. Globalisation eroded national borders, while “hypercapitalism” delivered concentrations of wealth not witnessed since 1914. In the context of post-socialist ideological cynicism

Free and Open

Many years later, the iPad was good at the simple media consumption convenience part of the idea, but fell quite flat in many ways with regard to the two larger ideas of (a) user authoring of meta-media, and (b) an environment to help children learn powerful ideas by making and sharing them. The latter was not in Apple’s goals at all: users (even children) were forbidden to make actively programmable things on the iPad and share them on the Internetthere was no stylus for drawing with the machine, and years later when one finally was added they omitted a place to store it (!) — and even still they did not include proximity sensing (compare with the Wacom Cintiq).fluent typing was not addressed in the design (we knew in 1968 that even with a touch-screen, a physical keyboard would almost certainly be needed). It is somewhat bizarre (even in this bizarre world) that Microsoft took more into account in its physical designs for the Surface …most importantly, Apple made no effort to help children by funding curriculum development, how to help them learn, etc. Steve’s earlier “Wheels for the Mind” ideal was long gone: buried in simple consumerism.So: don’t confuse the Dynabook idea with the physical resemblance to the iPad. The latter has thousands of times the capacity of what I had in mind, but its conception is thousands of times more meager.

table_row_count_estimate.csv Quickly see the number of rows in each table. table_sizes.csv See table "size on disk" as reported by Postgres stats. (Actual size is smaller.) writes_tables.csv Show the number of writes for each table.

fdptest

Total number of tables

Table of contents

Art / Earth / Tech

very skillfully illustrate the ideas with stories, walk through complex ideas and retain a balanced tone

but a very good academic, balanced tone from Gardner and Tetlock

Superforecasting

What a lovely time to shove an ad in someone's face.

This is why we postulate to have a central declaration file (as in YAML or JSON) per data asset, capturing all these properties required to run a generalized task (carried out by a custom operator). In other words, operators are designed in a generic way and receive the name of a data asset, from which they can grab its declaration file and learn how to parameterize and carry out the specific task.

What I Think Should Be Done  For the previously explained reasons, I believe that capitalism is a fundamentally sound system that is now not working well for the majority of people, so it must be reformed to provide many more equal opportunities and to be more productive. To make the changes, I believe something like the following is needed.  Leadership from the top. I have a principle that you will not effect change unless you affect the people who have their hands on the levers of power so that they move them to change things the way you want them to change. So there need to be powerful forces from the top of the country that proclaim the income/wealth/opportunity gap to be a national emergency and take on the responsibility for reengineering the system so that it works better. Bipartisan and skilled shapers of policy working together to redesign the system so it works better. I believe that we will do this in a bipartisan and skilled way or we will hurt each other. So I believe the leadership should create a bipartisan commission to bring together skilled people from different communities to come up with a plan to reengineer the system to simultaneously divide and increase the economic pie better. That plan will show how to raise money and spend/invest it well to produce good double bottom line returns. Clear metrics that can be used to judge success and hold the people in charge accountable for achieving it. In running the things I run, I like to have clear metrics that show how those who are responsible for things are doing and have rewards and punishments that are based on how these metrics change. Having these would produce the accountability and feedback loop that are required to achieve success. To the extent possible, I’d bring that sort of accountability down to the individual level to encourage an accountability culture in which individuals are aware of whether they are net contributors or net detractors to the society, and the individuals and the society make attempts to make them net contributors.   Redistribution of resources that will improve both the well-beings and the productivities of the vast majority of people. As an economic engineer, naturally I think about how money might be obtained from taxes, borrowing, businesses, and philanthropy, and how it would flow to affect prices and economies. For example, I think about how a change in personal tax rates might occur and how changes in them relative to corporate tax rates would affect how money would flow, and how changes in tax rates in one location relative to another location would drive flows and outcomes in them. I also think a lot about how the money raised will be spent—e.g., how much will be spent on programs that will improve both social and economic outcomes, and how much will be redistributive. Such decisions would of course be up to the people on the bipartisan commission and the leadership to decide and are way too complicated an engineering exercise for me to opine on here. I can, however, give my big picture inclinations. Above all else, I’d want to achieve good double bottom line results. To do that I’d:

Policy makers pay too much attention to budgets relative to returns on investments. For example, not spending money on educating our children well might be good from a budget perspective, but it’s really stupid from an investment perspective

The pursuit of greater profits and greater company efficiencies has also led companies to produce in other countries and to replace American workers with cost-effective foreign workers, which was good for these companies’ profits and efficiencies but bad for the American workers’ incomes.

The pursuit of profit and greater efficiencies has led to the invention of new technologies that replace people, which has made companies run more efficiently, rewarded those who invented these technologies, and hurt those who were replaced by them. This force will accelerate over the next several years, and there is no plan to deal with it well.

FREDERIC LALOUX

Reinventing Organisations

We are a strategic discovery, design and development lab working to transition society in response to technological revolution and climate breakdown.

EIT Climate-KIC is a Knowledge and Innovation Community (KIC), working to accelerate the transition to a zero-carbon economy. Supported by the European Institute of Innovation and Technology, we identify and support innovation that helps society mitigate and adapt to climate change. We believe that a decarbonised, sustainable economy is not only necessary to prevent catastrophic climate change, but presents a wealth of opportunities for business and society.

About InfluenceMap InfluenceMap empowers investors, corporations, the media and campaigners with data-driven and clearly communicated analysis on critical issues associated with climate change and the energy transition. Our flagship platform is the world's leading analysis of how companies and trade associations impact climate-motivated policy globally.

Our Deep Demonstrations are just that: Eight large-scale pan-European initiatives made up of a co-ordinated portfolio of projects designed to trigger systems change. These initiatives—from Just Transformations to Resilient Regions and Long-termism—follow our systems innovation methodology, in which we collaborate with design partners on ambitious systems-level challenges that are identified by “challenge owners” such as city mayors, regional leaders, government ministers, citizen leaders and CEOs of major companies. While most of the Deep Demonstrations are in the early stages, the work we have done with Slovenia shows what we can accomplish. The Slovenian government has embraced the radical, holistic approach that underpins our Deep Demonstrations, and their parliament has passed a motion adopting our proposal to become a fully circular economy. This collaboration is a part of our overall vision for the Deep Demonstration model—to trigger rapid decarbonisation and enhanced resilience across Europe.

To understand the High Line’s effect on surrounding property prices, we analysed publicly available valuation data from NYC’s Department of Finance, and cross-referenced it with property sales data for blocks and individual plots (a detailed methodology is available 👉 here). This meant we could track how the values of surrounding properties have changed since the High Line’s arrival.What’s interesting is that if we group the properties in bands roughly one kilometre wide from the High Line you start to see that between 2007 (when construction started) and 2018 (when the data ends), properties closer to the High Line experienced a greater value increase on average than those further away. So the mean property value uplift for houses within 1km of the High Line was actually 92% more than the Manhattan mean. Or to put it another way — if you owned an apartment in that 1km, you earned on average about $67,000 a year from the uplift alone. 🤑

Perhaps the greatest obstacle to redrawing the political map of the United States in this way is our bipartisan tradition of worshiping the founders. For many of the framers of the Constitution, nothing was more disturbing than the possibility that the Union would fracture into three or four smaller republics, or regional “sub-confederations.”

These videos are part of a new, free online course on income inequality in the United States available through Khan Academy.

Daniel Markovits, author of “The Meritocracy Trap,” estimates there are about one million of these workers in America today. They work really hard, are really productive and earn a lot more. In the mid-1960s, profits per partner at elite law firms were less than five times a secretary’s salary. Now, Markovits notes, they are over 40 times.

I chose to go to Compton and Watts for a specific reason, which offers a way forward. Harvard economist Raj Chetty recently led a study that showed that though these two neighborhoods are demographically similar and only 2.3 miles apart, 44 percent of the black men who grew up in Watts were incarcerated on April 1, 2010, compared with only 6.2 percent of the black men who grew up in families with similar incomes in Central Compton. Similarly, social mobility was much lower in Watts than in Compton.Why are some neighborhoods, including some in Compton, able to give their kids better chances in life despite so many disadvantages? Chetty points to several factors: better schools, more fathers present in the neighborhoods and more cohesive community organizations.I found all those things in my reporting in Compton — and something else. Watts is part of Los Angeles. Compton is its own city with its own mayor. I met a lot of great people in Watts, but Compton has more civic infrastructure — community groups and locally controlled government agencies. Compton has a lot of homegrown civic reformers, like Rafer Owens, who is a deputy Los Angeles County sheriff and pastor at a Baptist church. There’s also a mentality: We have faith in our ability to take care of ourselves; only people in the neighborhood really know what’s going on.

But here’s the situation: The information economy rains money on highly trained professionals — doctors, lawyers, corporate managers, engineers and so on.

Did you marry before having kids and raise your kids in a two-parent home? The children of the well educated are now much more likely to grow up in stable families, and those differences in family structure explain 32 percent of the growth of family income inequality since 1979.

We’re not talking about small numbers here. According to estimates by the Rockefeller Institute, from 2015 to 2018 Kentucky — which pays relatively little in federal taxes, because it’s fairly poor, but gets major benefits from programs like Medicare and Social Security — received net transfers from Washington averaging more than $33,000 per person. That was 18.6 percent of the state’s G.D.P.

The big thing about the Web isn’t the technology, it’s that it’s the first-ever platform without a vendor (credit for first pointing this out goes to Dave Winer). From that follows almost everything that matters, and it matters a lot now, to a huge number of people. It’s the only kind of platform I want to help build.

The iPhone vision of the mobile Internet’s future omits controversy, sex, and freedom, but includes strict limits on who can know what and who can say what. It’s a sterile Disney-fied walled garden surrounded by sharp-toothed lawyers. The people who create the apps serve at the landlord’s pleasure and fear his anger. I hate it. I hate it even though the iPhone hardware and software are great, because freedom’s not just another word for anything, nor is it an optional ingredient.

There now is a loud call for more “ethics in design”.

Focusing on the user is without any doubt a positive development for our industry. Design should serve humans. Measuring results and verifying assumptions may have been a painful process for the old masters. Life is change. Only, how much of what makes us human is truly measurable and verifiable? How do we measure friendship? By the number of replies per month? By the length of replies? With computer linguistics? How do we measure usefulness? Lots of page views? Few page views? Stickiness? Number of Subscriptions? How do we measure trust? By the number of likes? Retweets? Comments? How do we measure truth?

The field of transition research has experienced exponential growth over the last fifteen years and has had a significant impact, socio-technical. It has produced a wide range of analytical concepts along with science based interventions. It has however also become to a certain extent locked-in into a focus on socio-technical regimes in the predevelopment phase.

Over the coming years, we will work with local and national governments in these processes seeking to radically alter their dominant way of thinking and working and to experimentally develop new distributed models of non-linear systemic governance

From my perspective this would at least require a deep change in how we measure value, define labor and organize the financial system.

My personal focus will be on further developing the understanding, idea and practice of governance panarchy in light of the New Transformation. How can we reconceptualize the role of government and the state, how to reinvent the democratic system and how to reorganize our societies within social, economic and ecological boundaries. The basic idea of the three governance mechanisms I presented (bottom-up, top-down and phase-out) I will take as starting points to develop further a theoretical conceptualization of governance panarchy in the context of societal transitions addressing the period of instability, chaos, conflict, disruption and, hopefully, emergence of new sustable regimes. I aim to do so by working in a transdisciplinary way with proactive and transformative actors within government, business, science and civil society.

The majority of scholars in the social science domains seem to have abandoned the idea that there are identifiable patterns of change and broader explanations and therefore prediction and ex ante analysis are impossible.

As the transition we identified would predom-inantly require a shift in thinking and practices, the discussions were designed to help professionals and young people come to a new understanding and practice. To put it bluntly: the professionals needed to find out themselves that they were in part sustaining the problem. In the past couple of months there has been some progress on developing this shared understanding, formulating a corresponding vision and identifying some breakthrough projects. However, it is also becoming clear that the transitional problem is persistent and that the regime itself is very robust.

Our conclusion was that the field of professionals themselves are in this way unable to deal with the problem, which is created earlier on (in their home situations, in the professional youth care, on the streets) and that the reflex to organize solutions has mainly led to an increasing field of professionals and increasing systemic costs with no apparent impact on the quality of life of the young people involved. Also there is a constant competition between professional organizations and interest groups for (small) subsidies to conduct ‘innovative projects’ in some sort of search for the holy (institutional) grail for fighting youth homelessness.

The original principles for transition governance (such as long-term thinking, a focus on experimentation and learning, selective participation and dealing with systemic uncertainties) are derived from the understanding of social change as systemic, non-linear and complex.

Transition scenarios

Three governance mechanisms that need to come together to seize transition points.

ince the 1970s alternative currencies, renewable (energy) technologies, local democracies and sustainable community initiatives started to appear. For long these were small, expensive and often ridiculed as too alternative. But over time and with experience they grew, developed and matured. By now, many of these alternatives are starting to touch mainstream, from urban gardens and farms to energy producing buildings and from renewable energy cooperatives to credit unions and collective health care insur-ances. As I will argue such examples are part of a more fundamental underlying and structural process of transformation in which we shift from the drivers of modernity towards a new set of drivers.

Environmental policies much like the Sustainable Development discourse have become part of these established regimes and have primarily served to optimize these regimes making them a bit less unsustainable. I thus come to conclude that Sustainable Development itself has become part of the problem. The currently dominant regimes based on the foundations of modernity, are systemically unsustainable in a fundamental way.

The two cases are to me clear examples of the ́sustainability lock-in’: we seem to be caught in a vicious cycle of optimizing and inherently unsustainable system which closes down the space for the development of inherently better alternatives

business and markets were as free as possible.

I argue that the development of the modernistic regimes part of the Great Transformation started to stabilize around the 1960s. At the same time we can see that the New Transformation started to develop in niches. Already from the early 1960s, people started to become aware of the inherent unsustainability of our systems of consumption and production, in terms of environmental impact, economic equity and social justice. Ulrich Beck has previously pointed at this process in which societies become increasingly aware of negative externalities and start to deal with these and called this ‘reflexive modernisation’ (Beck 1994). The established regimes however predominantly focused on optimization, improvement and efficiency to reduce negative impacts. Over time however these regimes, because of their focus on optimization, became increasingly locked-in, adding to the persistency as well as their systemic vulnerability. Simultaneously the understanding of the persistent nature of our problems along with the alternative technologies and practices slowly ripened and matured. We are now in a period where dominant regimes become fundamentally challenged by these alternatives, creating the contexts for tensions, conflicts, surprises and, ultimately, disruptive systemic changes.

Most of the progress achieved in the Great Transformation has been powered by central planning, fossil resources and linear thinking. As much as social struggles and economic innovations have contributed to progress and growth, I argue that it is the triplet of central control, fossil resources and linear thinking that is the driving engine of modernity.

Understanding the Great Transformation as a family of socio-technical transitions.

Only through continuous dialectic processes of choice and resistance does society change, not in an endless gradual but in a shock-wise and non-linear way.

The transition perspective offers a coherent and descriptive explanation to such processes of seem-ingly coincidental change

his process has been described as the industrial revolution, modernization or ́the great transformation ́(Polanyi 1944).

The .ORG Stewardship Council will haveauthority to provide independent advice on anda binding right tovetomodifications proposed by Registry Operator to Registry Operator’s policies in the .ORG domain name space regarding: (x) censorship and freedom of expression; and (y) use of .ORG registrant and user data (the “Designated Policies”), in each case in accordance with the .ORG Stewardship Council charter (the “Charter”). Notwithstanding the foregoing, Registry Operator reserves the right at all times to ensure compliance in its sole judgment with applicable laws, policies and regulations.

Affordability.As of the Effective Date (June 30, 2019), the price Registry Operator charges to ICANN-accredited registrars for .ORG initial domain name registrations or renewal(s) of domain name registrations is US$9.93 (the "Service Fee"). At all times during the period ending eight (8) years following the Effective Date, the maximum allowable Service Fee (the “Applicable Maximum Fee”) that Registry Operator may charge to registrars for .ORG initial domain name registrations or renewal(s)

An intellectual prodigy as a child, Wilber was a Doctoral student at Duke University in biology when he quit his program in order to, as he put it, “sit in a room by myself and stare at a wall for five years.” He then went on a binge of studying eastern spirituality, religion, and psychology.

Ken Wilber is the smartest man you’ve never heard of.

Wilber’s story is a cautionary tale. His intellectual understanding was immense, as much as I’ve ever come across in a single person. He also tapped into some of the farthest reaches of consciousness, spiritual or not, that humans have self-reported. I do believe that. But ultimately, he was done in by his pride, his need for control and, well, ironically his ego.The point is, if Wilber can succumb to it, any of us can. No one is immune. No matter how brilliant and how “enlightened” we are, we’re all animals.

Following Wilber online, the conversation seemed to only become more and more insular. With an onslaught of problems in the world crying out for an integral perspective and solution — terrorism, the Iraq War, climate change, world hunger, financial crises — the silence coming from the Integral crowd was deafening. Major global and social issues were often only referred to in passing as descriptors for a certain level of consciousness development with the overarching implication being that “they” are not as highly developed as “we” are.We’re “second-tier” thinkers. We’re going to change the world… as soon as we’re done talking about how awesome and “second-tier” we are.Instead, most conversations involved esoteric spiritual topics, impulsive self-expressionism, and re-explaining the integral model in 4,102 different ways. For a philosophy based on including and integrating as much as possible, its followers sure expressed it by forming a nicely-sealed bubble around themselves.

Wilber bragged in an interview that he never planned anything at Integral Institute

brilliant mind does not necessarily make a brilliant leader.

What Wilber taught me is that no depth of spiritual experience can negate our physical and primal drives for power, lust, and validation. As primates, we’re wired to seek someone to look up to as well as to be looked up to by others. And that’s true whether we’re experiencing Godhead or bodhisattva or not. It’s inescapable.

As humans, we have a tendency to cling to ideologies. Any positive set of beliefs can quickly turn malevolent once treated as ideology and not an honest intellectual or experiential pursuit of greater truth.

But what he seems to have missed is that worshipping consciousness development itself, Wilber’s so-called “second-tier” thinking, leads to the same disastrous repercussions Wallace warned of: vanity, power, guilt, obsession.No one is immune.

The second problem with Paxos is that it does not pro-vide a good foundation for building practical implemen-tations. One reason is that there is no widely agreed-upon algorithm for multi-Paxos. Lamport’s descriptionsare mostly about single-decree Paxos; he sketched possi-ble approaches to multi-Paxos, but many details are miss-ing.

Unfortunately, Paxos has two significant drawbacks.The first drawback is that Paxos is exceptionally diffi-cult to understand. The full explanation [15] is notori-ously opaque; few people succeed in understanding it, andonly with great effort. As a result, there have been severalattempts to explain Paxos in simpler terms [16, 20, 21].These explanations focus on the single-decree subset, yetthey are still challenging. In an informal survey of atten-dees at NSDI 2012, we found few people who were com-fortable with Paxos, even among seasoned researchers.We struggled with Paxos ourselves; we were not able tounderstand the complete protocol until after reading sev-eral simplified explanations and designing our own alter-native protocol, a process that took almost a year.

As you can see the CSV parser is 20x faster and performs 75% fewer allocations.

Google Ad LibraryScroll down Google releases their political ads archive as a public dataset. We describe how we download political ads from Google Cloud Services on our methods page, or scroll down to see our data collection log. We did not experience any technical issue with the dataset or the download process.

This page documents our efforts to track political advertising and produce an ad transparency report for the 2019 European Parliament Election.We attempted to download a copy of the political ads on a daily basis using the Facebook Ad Library API and the Google Ad Library, starting on March 29 and May 11 respectively, when the two companies released their political ads archive. We provide this data collection log, so that external researchers, journalists, analysts, and readers may examine our methods and assess the data presented in our reports.Facebook Ad Library APIScroll down Facebook provides an Application Programmable Interface ("API") to authorized users who may search for ads in their archive. However, due to the inconsistent state of the Facebook Ad Library API, our methods to scan and discover ads must be adapted on a daily and sometimes hourly basis — to deal with design limitations, data issues, and numerous software bugs in the Facebook Ad Library API.Despite our best efforts to help Facebook debug their system, the majority of the issues were not resolved. The API delivered incomplete data on most days from its release through May 16, when Facebook fixed a critical bug. The API was broken again from May 18 through May 26, the last day of the elections.We regret we do not have reliable or predictable instructions on how to retrieve political ads from Facebook. Visit the methods page for our default crawler settings and a list of suggested workarounds for known bugs, or scroll down to see our log.In general, we encountered three categories of issues with the Facebook Ad Library API. First, software programming errors that cripple a user's ability to complete even a single search including the following bugs:

We believe open collaboration is essential for progress

In recent years, computing has both expanded as a field and grown in its importance to society. Similarly, the research conducted at Google has broadened dramatically, becoming more important than ever to our mission. As such, our research philosophy has become more expansive than the hybrid approach to research we described in our CACM article six years ago and now incorporates a substantial amount of open-ended, long-term research driven more by scientific curiosity than current product needs.

In the meantime, I asked Fries, if Shell is serious about transition, then couldn’t it voluntarily speed it up by leaving some of its wells fallow, constraining oil output and thereby driving the price relative to renewables higher, faster? Sure, it would have to take some losses in the short term, but we’re talking about the future of the planet here. He dismissed the idea, telling me it’s important not to artificially withhold supply, which would introduce price shocks that could turn public opinion against environmentalist policy. Besides, it would only end up sending money to the Saudis anyway. “We’re going to get as much out of [oil and gas] for as long as we can,” he said. “That’s an extremely frightening thing for you to say,” I said. “It doesn’t mean every drop,” he said, failing to reassure me.

In the corporate sector, there’s still faith at the top that economic incentives and profit-seeking behavior can manage the crisis that capitalism has wrought. In such thinking, climate change is like a redux of the hole in the ozone layer: potentially bad but solvable with the tools on hand and without real changes to our lifestyles. Fries estimates that we’ll be able to cost-effectively fill two-thirds of world energy demand with clean sources within 20 years. (That’s ten years more optimistic than the optimistic scenario of the International Renewable Energy Agency, an intergovernmental organization mandated to propagate optimistic scenarios about renewable-energy transition.)

The great Harvard marketing professor Theodore Levitt used to tell his students, “People don’t want to buy a quarter-inch drill. They want a quarter-inch hole!” Every marketer we know agrees with Levitt’s insight. Yet these same people segment their markets by type of drill and by price point; they measure market share of drills, not holes; and they benchmark the features and functions of their drill, not their hole, against those of rivals. They then set to work offering more features and functions in the belief that these will translate into better pricing and market share. When marketers do this, they often solve the wrong problems, improving their products in ways that are irrelevant to their customers’ needs.

Standardized test scores improved dramatically. In 2006, only 10% of Noyes' students scored "proficient" or "advanced" in math on the standardized tests required by the federal No Child Left Behind law. Two years later, 58% achieved that level. The school showed similar gains in reading. Because of the remarkable turnaround, the U.S. Department of Education named the school in northeast Washington a National Blue Ribbon School. Noyes was one of 264 public schools nationwide given that award in 2009. Michelle Rhee, then chancellor of D.C. schools, took a special interest in Noyes. She touted the school, which now serves preschoolers through eighth-graders, as an example of how the sweeping changes she championed could transform even the lowest-performing Washington schools. Twice in three years, she rewarded Noyes' staff for boosting scores: In 2008 and again in 2010, each teacher won an $8,000 bonus, and the principal won $10,000. A closer look at Noyes, however, raises questions about its test scores from 2006 to 2010. Its proficiency rates rose at a much faster rate than the average for D.C. schools. Then, in 2010, when scores dipped for most of the district's elementary schools, Noyes' proficiency rates fell further than average.

Atlanta’s rampant test manipulation amplified calls for nationwide education reform. Seven years after the Atlanta Journal-Constitution first reported on testing problems, policymakers have failed to make significant progress toward changing the way students take standardized tests and how teachers interpret those scores. In fact, the problem has worsened, resulting in documented cheating in at least 40 states, since the APS cheating scandal first came to light. “Atlanta is the tip of the iceberg,” says Bob Schaeffer, public education director of FairTest, a nonprofit opposed to current testing standards. “Cheating is a predictable outcome of what happens when public policy puts too much pressure on test scores.” Some experts, including Schaeffer, point to the No Child Left Behind Act of 2001 as a source of today’s testing problems, though others say the woes predated the law. Then-president George W Bush, who signed the measure in January 2002, aimed to boost national academic performance and close the achievement gap between white and minority students. To make that happen, the law relied upon standardized tests designed to hold teachers accountable for classroom improvements. Federal funding hinged on school improvements, as did the future of the lowest-performing schools. But teachers in many urban school districts already faced enormous challenges that fell outside their control – including high poverty, insufficient food access, and unstable family situations. Though high-stakes testing increased student achievement in some schools, the federal mandate turned an already-difficult challenge into a feat some considered insurmountable. The pressure led to problems. Dr Beverly Hall, the former APS superintendent who was praised for turning around student performance, was later accused of orchestrating the cheating operation. During her tenure, Georgia investigators found 178 educators had inflated test scores at 44 elementary and middle schools.

Atlanta public schools. The urban school district has already suffered one of the most devastating standardized-testing scandals of recent years. A state investigation in 2011 found that 178 principals and teachers in the city school district were involved in cheating on standardized tests. Dozens of former employees of the school district have either been fired or have resigned, and 21 educators have pleaded guilty to crimes like obstruction and making false statements.

The differences between the report and Mr. Barr’s description of it “cause the court to seriously question whether Attorney General Barr made a calculated attempt to influence public discourse about the Mueller report in favor of President Trump despite certain findings in the redacted version of the Mueller report to the contrary,” wrote Judge Walton, an appointee of President George W. Bush.

Although he sat as a crossbencher, Neill’s stance on the EU was more in keeping with eurosceptic Conservatives. He argued forcefully that the 1972 European Communities Act, by which Britain joined the EEC, had overthrown constitutional law and tradition. Later, he was highly critical of the European Court, saying that its judgements were designed to advance the cause of greater European integration. “A court with a mission is a menace,” wrote Neill. “A supreme court with a mission is a tyranny.” In 1996 the home secretary Michael Howard adopted the phrase when he argued that British courts should be forbidden from enforcing European Community law.

Data extraction is a piece of a larger puzzle called data integration (getting the data you want to a single place, from different systems, the way you want it), which people have been working on since the early 1980s.

I also like that this architecture highlights the problem of reprocessing data. Reprocessing is one of the key challenges of stream processing but is very often ignored. By “reprocessing,” I mean processing input data over again to re-derive output. This is a completely obvious but often ignored requirement. Code will always change. So, if you have code that derives output data from an input stream, whenever the code changes, you will need to recompute your output to see the effect of the change.

I like that the Lambda Architecture emphasizes retaining the input data unchanged. I think the discipline of modeling data transformation as a series of materialized stages from an original input has a lot of merit. This is one of the things that makes large MapReduce workflows tractable, as it enables you to debug each stage independently. I think this lesson translates well to the stream processing domain. I’ve written some of my thoughts about capturing and transforming immutable data streams here.

ParDofor generic parallel processing. Each input ele-ment to be processed (which itself may be a finite col-lection) is provided to a user-defined function (calledaDoFnin Dataflow), which can yield zero or more out-put elements per input.

When describing infinite/finitedata sets, we prefer theterms unbounded/bounded over streaming/batch, becausethe latter terms carry with them an implication of the useof a specific type of execution engine. In reality, unboundeddatasets have been processed using repeated runs of batchsystems since their conception, and well-designed streamingsystems are perfectly capable of processing bounded data.From the perspective of the model, the distinction of stream-ing or batch is largely irrelevant, and we thus reserve thoseterms exclusively for describing runtimeexecution engines.

Unbounded, unordered, global-scale datasets are increas-ingly common in day-to-day business (e.g. Web logs, mobileusage statistics, and sensor networks).

Visionary companies are guided by a mission, not just a sustainable business model, and seek out challenges that can help move humanity forward. We have the opportunity to do that at Stack Overflow, to realize our profound mission of helping write the script of the future by serving developers and technical workers. Let’s answer the most important questions in this great new era of technology, together.

Delivering on such a 2°C emission pathway cannot be reconciled with the repeated and high-level claims that in transitioning to a low-carbon energy system “global economic growth would not be strongly affected”7.

3) The Report proposes a headline carbon budget of 1,000 billion tonnes of carbon dioxide (1000 GtCO2) for the period 2011 to 2100 and for a 66% chance, or better, of remaining below a 2°C rise18.

The analysis within this Commentary makes no allowance for carbon budgets being increased through the adoption of ‘geo-engineering’ technologies, specifically those delivering so-called negative emissions. Such technologies are ubiquitous in 2°C scenarios9,10, despite their remaining at little more than the conceptual stage of development. However, whilst speculative negative emissions are de rigueur, similarly imprecise Earth system processes (but with the potential to reduce the available budgets) are seldom included in quantitative scenarios. The relative importance of negative emissions and Earth-system processes for the size of the available carbon budget varies across the spectrum of temperatures being considered. Yet until both can be adequately and robustly quantified their widespread inclusion within quantitative emissions pathways should be avoided. A small suite of 2°C scenarios may, of course, assume the successful uptake of negative emissions (or further positive feedbacks), but such scenarios should be in the minority and not dominate the outputs from across the IAM community.

While our differences are often rooted in divergent views, that does not mean we cannot find common ground By acknowledging and respecting the values that animate our beliefs, we can begin to restore a sense of respect and unity

At the root of those disagreements are differences in core beliefs―the underlying psychological architecture that governs what we value and how we see the world

They want to return to the mutual good faith and collaborative spirit that characterize a healthy democracy

oversimplified story of polarization would make us believe

We asked subjects a series of questions designed to assess how concerned they were with each of the five moral foundations in their moral judgments. Our results showed strong distinctions according to the various tribes. Progressive Activists and Traditional and Passive Liberals tended to care more about Harm and Fairness than the other foundations, while right-leaning groups such as Traditional and Devoted Conservatives cared about all five foundations.

84 percent of Devoted Conservatives believe Americans do not take terrorism seriously enough

For instance, 69 percent of Americans believe that we have become too sensitive to issues of race, and a near-unanimous 85 percent think that "race should not be a factor" in college admissions.

Core Beliefs and Demographics Tribe membership (pictured here: Progressive Activists and Devoted Conservatives) predicts how people think about political issues better than standard categories (such as "Liberal" or "Republican")

Devoted Conservatives (6 percent of the population) are deeply engaged with politics and hold strident, uncompromising views. They feel that America is embattled, and they perceive themselves as the last defenders of traditional values that are under threat.

Traditional Conservatives (19 percent of the population) tend to be religious, patriotic, and highly moralistic. They believe deeply in personal responsibility and self-reliance.

Moderates (15 percent of the population) are engaged in their communities, well informed, and civic-minded. Their faith is often an important part of their lives. They shy away from extremism of any sort.

The Politically Disengaged (26 percent of the population) are untrusting, suspicious about external threats, conspiratorially minded, and pessimistic about progress. They tend to be patriotic yet detached from politics.

Passive Liberals (15 percent of the population) tend to feel isolated from their communities. They are insecure in their beliefs and try to avoid political conversations. They have a fatalistic view of politics and feel that the circumstances of their lives are beyond their control.

Traditional Liberals (11 percent of the population) tend to be cautious, rational, and idealistic. They value tolerance and compromise. They place great faith in institutions.

Progressive Activists (8 percent of the population) are deeply concerned with issues concerning equity, fairness, and America's direction today. They tend to be more secular, cosmopolitan, and highly engaged with social media.

Traditional Liberals place more value in authority and loyalty, are less likely to rate their ideological identity above being American, and are more likely to see political correctness as a problem. “Progressive Activists

Dixon told me about one of the researchers who helped conduct follow-up interviews, a young woman with left-wing views. “She went in to an interview with a Traditional Conservative girded for battle, and after an hour she felt a transformational experience,” he said. In asking about matters like family values and threats to the community, he said, “she had an appreciation that their views made sense.”

For example, eighty-one per cent of those interviewed believe that racism is a serious problem, but eighty-five per cent think that race should not be a factor in college admissions.

The tribes in the report are different from the rigid and unchanging partisan monoliths of our national political debate. (For this reason, perhaps More in Common should have used a term other than “tribes.”)

More in Common found that “tribal membership predicts differences in Americans’ views on various political issues better than demographic, ideological, and partisan groupings.” In other words, whether or not you think creativity is more important than good behavior in children is a better indicator of your political views than is your gender, your race, your income, or your party affiliation. “Once we have the seven segments, their views on issues are highly correlated,” Tim Dixon, an Australian political activist and a founder of More in Common, told me. He added, “We have too much opinion research and not enough value research.”

The agenda of power at any price gave conservatism a cynical, even nihilistic strain—Goldwater and Reagan were honest zealots by comparison—but the essential ingredients had been there all along. Max Boot, in his book, makes the point unflinchingly: “Upon closer examination, it’s obvious that the whole history of modern conservatism is permeated with racism, extremism, conspiracy-mongering, ignorance, isolationism, and know-nothingism. Even those who were not guilty of these sins too often ignored them in the name of unity on the right.” Trump is the movement’s darkest realization, not its betrayal.

As President, Reagan subdued his own extremism, but, no matter how Republicans governed, every four years they found new ways—from the Southern strategy to welfare queens to Willie Horton—to divide the right kind of Americans from the wrong kind. When Reagan left office and the Cold War ended, conservatism lost its best spokesman and its organizing principle. The movement’s subsequent leaders—Gingrich, Rush Limbaugh, Jerry Falwell, Jr., Ann Coulter, Sarah Palin, Ted Cruz—pursued power, celebrity, and their enemies on the left without a trace of Reagan’s optimistic gloss or William F. Buckley’s intellectual dash. They pushed conspiracy theories into the mainstream. They kept raising the bar of viciousness.

Its extremism goes straight back to Flake’s hero, Senator Barry Goldwater, who saw mainstream liberals as subversive socialists and opposed the Civil Rights Act of 1964 on states’-rights grounds. In that year’s Presidential election, Goldwater received potent support from the best-selling writers Phyllis Schlafly, whose book “A Choice Not an Echo” imagined cabals of liberal Republicans plotting against the Party’s base, and John Stormer, who, in “None Dare Call It Treason,” warned that pro-Communist élites were infesting American institutions. Reagan’s famous half-hour commercial for Goldwater described the welfare state as the path to totalitarianism. Apocalyptic thinking, conspiracy theories, and bigotry haunted the movement from the start.

Flake’s book “Conscience of a Conservative” goes further. It traces the Party’s corruption back to Newt Gingrich and his lieutenants, and then chronicles the dark rise of Trump. Flake sees the collapse in terms of failures of character on the part of Republican leaders: in the nineties, Gingrich brought the politics of total war to Washington, while Tom DeLay, of Texas, infected the Party with partisan corruption. Flake wants to rescue conservatism from Trumpism and purify it. He can’t grasp that the modern conservative movement always carried the seeds of its own destruction.

Tokay continued, “I expect you to defend the constitution as I did and you must sacrifice your career as a republican and go against your party if you want to be doing this job in 18 months.”

In February, the non-stop hostility led him to abandon his Facebook page. Children in his neighborhood came up and asked him, incredulously, “You work for Trump?” He tried to teach them about the separation of powers. The political danger came from the right, but the vitriol was more personal on the left. One man from West Chester, a banker, left a series of obscene voice mails: “Mr. Costello, impeach the motherfucker, or, if you don’t have the vertebrae to do that, resign”; “You fucking asshole! Pay for the tax cuts to corporations and to the one-per-cent wealthy, and cut my benefits that I paid into the system for out of my wages? You are inhumane, other than being a fucking idiot. . . . And if I go bankrupt you’re going down with me, and your family will go down with me, too.”

If party discipline and political self-interest don’t fully explain the behavior of the congressional majority, something else must—tribal cohesion, fear of being shunned. Costello refused to quote Republican colleagues by name to me, or to say a bad word about any of them. Although he was appalled by right-wing attacks on the intelligence agencies and on the Mueller investigation, when I asked him about the chief conspiracy-monger—Devin Nunes, the chairman of the House Intelligence Committee, who has lobbed wild accusations about an F.B.I. plot to destroy Trump—Costello said that he didn’t know enough about Nunes’s claims to assess their validity. He simply said, “I know Devin, I like Devin.” Meanwhile, he sharply criticized such Democrats as Senators Cory Booker and Kirsten Gillibrand for “prejudging” the guilt of Brett Kavanaugh before Christine Blasey Ford had even testified.

However, there are some great techniques available to do exactly that. If you’re highly constrained on time and budget, try a Minimum Viable Ethnography, pioneered by user research expert Erika Hall of Mule Design.

The Extreme team used this insight to inform their decisions about the product’s direction. Instead of a cheaper incubator (the initial concept, but likely ineffective given the evidence) they decided to design something to help babies at home: a portable incubator, much like a tiny, heated sleeping bag, which they named Embrace.

Given the overhang of long-established customs and institutions, ahypothesis may be offered concerning the comparison of China and Eu-rope in early modern times: In China, the economic and social develop-ments of the Late Imperial age were less innovations than continuations.Though comparable to Europe in bulk, activity, and sophistication, eigh-teenth-century China was at the end of a period of high civilization thathad begun in the Northern Song eight centuries before, while Europe ofthe Enlightenment was just embarking on a quite new phase of worldhistory. Or to put it another way, new things in China could arise onlywithin the inherited matrix of imperial autocracy and gentry society thatwould remain dominant throughout the nineteenth century. This is thefocus of our next section.

The written records of popular culture included awide range, from almanacs to the scriptures of religious sects. Localoperas or other dramas were organized and enacted at market towns orat the village level or often by lineages. But throughout the popular cul-ture dissident voices were not permitted to be heard and were destroyedif possible.

With this wide support of morality went the menace of the criminallaw and punishment of malefactors against morality, especially againstthe dynasty. This application of the law against evildoers or the merethreat of evildoing included the uninhibited investigation of people intheir households and personal lives and the use of judicial torture to en-courage confessions. The ankle squeezer used in court was stepped up soas to maximize pressure, and it could turn bones into jelly when skillfullyapplied. When in doubt as to what law had been violated, the magis-trate-judges could fall back on the statute against “doing what ought notto be done,” whatever it might have been.By these rewards and punishments it was hoped the common peoplecould be kept in the proper path. The punishment of relatives was a reg-ular part of the punishment of the criminal. The ancient device of groupresponsibility meant in effect guilt by association.In the theocratic Chinese state that extolled the emperor as Son ofHeaven, heterodoxy was perpetually guarded against. The strategic elitestratum was the local leadership that began with the roughly one millionlower gentry or holders of the first-level (shengyuanorjiansheng)de-grees, which did not qualify one for official appointment but conferred aprivileged status and opportunity to seek higher degrees. To these wereadded possibly five million male commoners, more or less, who hadachieved some amount of classical education. With their help, the indoc-trination of the common people was pursued by the elite as a Neo-Con-fucian duty.As an example let us cite the use of theSacred Edict (shengyu)of theKangxi Emperor issued in 1670 as 16 maxims for the guidance of dailyconduct. Each maxim seven characters long, they conveyed, as VictorMair (in Johnson et al., 1985) says, “the bare bones of Confucian ortho-doxy as it pertained to the average citizen.” After 1670 appeared com-mentaries, paraphrases, adaptations, and so on, in a considerable liter-ature. The idea of explicating classical texts in written colloquialversions seems to have begun in the Yuan dynasty.

In today’s media environment where citizens observe events directlywith less need but more supply of symbols, it is not easy to appreciate theimportance of ritual and ceremony in an earlier time. One basis of thegovernment of imperial China was the proper performance of ceremo-nies at all levels of society. The son kowtowed to his father as his fathermight do toward the emperor and his officials, for the essence of the civilorder was to differentiate the hierarchy of relationships. Proper conduct,it was hoped, externalized one’s inner values; but even in the absence ofinner feelings, one’s performance of ritual could provide a common for-mal bond with others. In this way the appearance of harmony could as-sure it. As Naquin and Rawski (1987) put it, “luanwas the disorder thatcould arise within the state, the community, the household or the indi-vidual when ethical norms and correct ritual were not followed. The de-sire to promote order and preventluanpermeated Chinese society fromtop to bottom.”

After the Ming collapse, Qing critics would later decry Wang’s influ-ence as too abstract, passive, and individual-centered. This contributedto the view that Ming learning had fostered a righteous morality overpractical technology. Neo-Confucian classical training in the schools ofboth Zhu Xi and Wang Yangming taught Ming officials to assert thatethical conduct was the root of good government, while technology wasa matter for craftsmen and inferiors.

In the domain of literati thinking, the ideas of the statesman-philosopherWang Yangming (Wang Shouren, 1472–1529) gained many adherentsand inspired scholars to follow a new bent in Neo-Confucianism. Wangwas a very competent scholar-official and general who suppressed rebel-lions over a number of years and also devoted himself to building up thelocal community through the use of the Community Compact(xiang-yue).This institution was one of Confucianism’s closest approaches torevivalism. As a philosopher, Wang pursued the idea of Zhu Xi’s contem-porary, Lu Xiangshan, in developing a less practice-centered and morecontemplative approach to moral training and self-cultivation. Wangtaught that the world of principle is a unity and lies within as well as out-side one. Therefore, one should learn to be guided by intuitive knowl-edge achieved through careful thought and meditation. This had Bud-dhist overtones. Wang’s famous insistence on the unity of theory andpractice really demanded, as Willard J. Peterson (1979) notes, “the unityofmoralknowledge andsocialaction.” Wang Yangming’s teaching hadwide influence in Japan as well as in China.

In short, anticommercialism and xenophobia won out, and China re-tired from the world scene. The military declined and bureaucrats ranthe show except when the big eunuch establishment, handling surveil-lance and investigation for the emperor, produced from time to time, un-der weak rulers, eunuch dictatorships that terrorized the scholars. Thecontradiction between Ming China’s superior capacity for maritime ex-pansion and conservative Neo-Confucian throttling of it suggests thatMing China almost purposely missed the boat of modern technologicaland economic development.This disparaging judgment comes out of the context of the late twen-tieth century, when technology and growth have created innumerabledisorders in all aspects of life all over the world without disclosing asyet the principles of order that may postpone the destruction of humancivilization. In time the self-contained growth of Ming China with its

I think this has the added benefit of making data warehousing ETL much more organizationally scalable. The classic problem of the data warehouse team is that they are responsible for collecting and cleaning all the data generated by every other team in the organization. The incentives are not aligned: data producers are often not very aware of the use of the data in the data warehouse and end up creating data that is hard to extract or requires heavy, hard to scale transformation to get into usable form. Of course, the central team never quite manages to scale to match the pace of the rest of the organization, so data coverage is always spotty, data flow is fragile, and changes are slow.

I'll give a little bit of the history to provide context. My own involvement in this started around 2008 after we had shipped our key-value store. My next project was to try to get a working Hadoop setup going, and move some of our recommendation processes there. Having little experience in this area, we naturally budgeted a few weeks for getting data in and out, and the rest of our time for implementing fancy prediction algorithms. So began a long slog. We originally planned to just scrape the data out of our existing Oracle data warehouse. The first discovery was that getting data out of Oracle quickly is something of a dark art. Worse, the data warehouse processing was not appropriate for the production batch processing we planned for Hadoop—much of the processing was non-reversable and specific to the reporting being done. We ended up avoiding the data warehouse and going directly to source databases and log files. Finally, we implemented another pipeline to load data into our key-value store for serving results. This mundane data copying ended up being one of the dominate items for the original development. Worse, any time there was a problem in any of the pipelines, the Hadoop system was largely useless—running fancy algorithms on bad data just produces more bad data. Although we had built things in a fairly generic way, each new data source required custom configuration to set up. It also proved to be the source of a huge number of errors and failures. The site features we had implemented on Hadoop became popular and we found ourselves with a long list of interested engineers. Each user had a list of systems they wanted integration with and a long list of new data feeds they wanted. ETL in Ancient Greece. Not much has changed.

Effective use of data follows a kind of Maslow's hierarchy of needs. The base of the pyramid involves capturing all the relevant data, being able to put it together in an applicable processing environment (be that a fancy real-time query system or just text files and python scripts). This data needs to be modeled in a uniform way to make it easy to read and process. Once these basic needs of capturing data in a uniform way are taken care of it is reasonable to work on infrastructure to process this data in various ways—MapReduce, real-time query systems, etc. It's worth noting the obvious: without a reliable and complete data flow, a Hadoop cluster is little more than a very expensive and difficult to assemble space heater. Once data and processing are available, one can move concern on to more refined problems of good data models and consistent well understood semantics. Finally, concentration can shift to more sophisticated processing—better visualization, reporting, and algorithmic processing and prediction. In my experience, most organizations have huge holes in the base of this pyramid—they lack reliable complete data flow—but want to jump directly to advanced data modeling techniques. This is completely backwards. So the question is, how can we build reliable data flow throughout all the data systems in an organization?

You don't hear much about data integration in all the breathless interest and hype around the idea of big data, but nonetheless, I believe this mundane problem of "making the data available" is one of the more valuable things an organization can focus on.

Data integration is making all the data an organization has available in all its services and systems.

Behind thislay the Confucian disdain for the military, which classed them evenlower than merchants. So deep-laid was this dislike that the militarywere excluded from the standard Confucian list of the four occupationalgroups or classes—scholar(shi),farmer(nong),artisan(gong),and mer-chant(shang).Derk Bodde (1991) tells us that this four-part division ofsociety was never put forward by Confucius or Mencius but first ap-peared probably among Legalist writers of late Zhou and early Han. Fortwenty-one centuries since that time, however, the four classes have beenstandard fare in the lore about China.

For the scholar-elite, Zhu Xi promoted academies. He had contactwith about 24 such unofficial institutions and taught 20 students in hisown. The object of this teaching was the individual, who must learn howto achieve his own grasp of morality and bear responsibility for hismoral self-cultivation in his effort to become a sage. Zhu hoped thatproper government, finally, would rest upon “universal self-disciplinebeginning with the ruler’s self-rectification.” This could be aided byscholars’ lectures to him (as part of court ritual) as well as by the subse-quent judgments of the court historians. In discussing moral questions,minister and emperor should talk as equals.