I want to select only some part of a dataset

APIs often differ from bulk download in their data format: following web conventions data APIs usually return the data in a standard format such as JSON (and can also provide various other formats e.g. XML)

Dirty

"Big Data" cycle

variety

"Big Data" was about waking up to the potential of data-driven insight and data-driven applications in business and society

The essence of "big data" was actually just data.

Section 6

Super news junkies

Table of Contents

Basically scale up to 1 and down to 0 because individual forecasts bias down / up at those points but mean need not …From Edge piece (see below)

https://pubsonline.informs.org/doi/abs/10.1287/deca.2014.0293 When aggregating the probability estimates of many individuals to form a consensus probability estimate of an uncertain future event, it is common to combine them using a simple weighted average. Such aggregated probabilities correspond more closely to the real world if they are transformed by pushing them closer to 0 or 1. We explain the need for such transformations in terms of two distorting factors: The first factor is the compression of the probability scale at the two ends, so that random error tends to push the average probability toward 0.5. This effect does not occur for the median forecast, or, arguably, for the mean of the log odds of individual forecasts. The second factor—which affects mean, median, and mean of log odds—is the result of forecasters taking into account their individual ignorance of the total body of information available. Individual confidence in the direction of a probability judgment (high/low) thus fails to take into account the wisdom of crowds that results from combining different evidence available to different judges. We show that the same transformation function can approximately eliminate both distorting effects with different parameters for the mean and the median. And we show how, in principle, use of the median can help distinguish the two effects

Indeed, extremizing gave regular forecaster teams a big enough boost to pass some superteams, and extremizing a large pool

note teams were 12 or more (I think)

8

But not all disturbances are equal. Remember that Keynes quotation about changing your mind in light of changed facts? It’s cited in countless books, including one written by me and another by my coauthor. Google it and you will find it’s all over the Internet. Of the many famous things Keynes said it’s probably the most famous. But while researching this book, I tried to track it to its source and failed. Instead, I found a post by a Wall Street Journal blogger, which said that no one has ever discovered its provenance and the two leading experts on Keynes think it is apocryphal. 7 In light of these facts, and in the spirit of what Keynes apparently never said, I concluded that I was wrong. And I have now confessed to the world. Was that hard? Not really. Many smart people made the same mistake, so it’s not embarrassing to own up to it. The quotation wasn’t central to my work and being right about it wasn’t part of my identity. But if I had staked my career on that quotation, my reaction might have been less casual. Social psychologists have long known that getting people to publicly commit to a belief is a great way to freeze it in place, making it resistant to change. The stronger the commitment, the greater the resistance.

Supersmart: are they super intelligent? No, but they are generally reasonably smart. Superquants: are the SFs just math geniuses? No, but they are all numerate and they have a good understanding of basic probability, including base rates etc. (something most of us don’t have). Supernewsjunkies: are SFs just good because they consume lots of information? Yes and no. It’s the quality and variety of what they consume; many of the good forecasters did not spend that much time reading material.

Illusions of knowledge: why we think we are better than we are and why to

“

2005

3. Observe how people solve the problem now (i.e. which job do they currently use).

E-mail signature template

issue

Meeting your Mentor / Buddy

The AET style guide

Github – https://www.github.com. GitLab – https://gitlab.com. HackMD – https://hackmd.io.

Gravatar