With approximately half of variance in cognitive task performance having non-g sources of variance, we believe that other traits may be important in explaining cognitive performance of both non-Western and Western groups.
Another important implication of the study.
Although we believe that this study establishes the presence of g in data from these non-Western cultures, this study says nothing about the relative level of general cognitive ability in various societies, nor can it be used to make cross-cultural comparisons. For this purpose, one must establish measurement invariance of a test across different cultural groups (e.g., Holding et al., 2018) to ensure that test items and tasks function in a similar way for each group.
This is absolutely essential to understanding the implications of the article.
This is because most authors usually did not report a plausible theoretical model for the structure of their observed variables, and there was often insufficient information for us to create our own plausible non-g models that could be compared with a theory of the existence of Spearman’s g in the data
The EFA vs. CFA question was a stickler for one peer reviewer, and I can understand why. When measurement is based on strong theory, then I believe that CFA is preferable to EFA. But that was rarely the case in these datasets.
Two peer reviewers raised the possibility that developmental differences across age groups could be a confounding variable because a g factor may be weaker in children than adults.
Colom also suggested this (see link above). The fact that three people independently had this concern that age could be a moderator variable is telling. I'm glad the peer reviewers had us do this post hoc analysis.
some of these data sets were collected by individuals who are skeptical of the existence or primacy of g in general or in non-Western cultures (e.g., Hashmi et al., 2010; Hashmi, Tirmizi, Shah, & Khan, 2011; O’Donnell et al., 2012; Pitchford & Outhwaite, 2016; Stemler et al., 2009; Sternberg et al., 2001, 2002). One would think that these investigators would be most likely to include variables in their data sets that would form an additional factor. Yet, with only three ambiguous exceptions (Grigorenko et al., 2006; Gurven et al., 2017), these researchers’ data still produced g.
This is particularly strong evidence for me. If g doesn't exist, these researchers would be the most likely ones to gather data to show that.
the strongest first factor accounted for 86.3% of observed variable variance
I suspect that this factor was so strong because it consisted of only four observed variables, and three of them were written measures of verbal content. All of the verbal cariables correlated r = .72 to .89. Even the "non-verbal" variable (numerical ability) correlates r = .72 to .81 with the other three variables (Rehna & Hanif, 2017, p. 25). Given these strong correlations, a very strong first factor is almost inevitable.
The weakest first factor accounted for 18.3% of variance
This factor may be weak because the sample consists of Sudanese gifted children, which may have restricted the range of correlations in the dataset.
The mean sample size of the remaining data sets was 539.6 (SD = 1,574.5). The large standard deviation in relationship to the mean is indicative of the noticeably positively skewed distribution of sample sizes, a finding supported by the much smaller median of 170 and skewness value of 6.297. There were 16,559 females (33.1%), 25,431 males (48.6%), and 10,350 individuals whose gender was unreported (19.8%). The majority of samples—62 of 97 samples (63.9%)—consisted entirely or predominantly of individuals below 18. Most of the remaining samples contained entirely or predominantly adults (32 data sets, 33.0%), and the remaining 3 datasets (3.1%) had an unknown age range or an unknown mix of adults and children). The samples span nearly the entire range of life span development, from age 2 to elderly individuals.
My colleague, Roberto Colom, stated in his blog (link below) that he would have discarded samples with fewer than 100 individuals. This is a legitimate analysis decision. See his other commentary (in Spanish) at https://robertocolom.wordpress.com/2018/06/01/la-universalidad-del-factor-general-de-inteligencia-g/
This information was not explicitly stated in either article, but the sample and community description makes it clear that the participants of these studies are the same people, though the sample sizes differ slightly (ns = 85 and 86). However, this redundancy did not produce any analysis problems because the correlation matrix in the Grigorenko et al. (2001) article was not positive definite.
The duplication of data across articles and the non-positive definite dataset have never been fully explained. In light of Sternberg's history of self-plagiarism (see link below), this is troubling.
https://medium.com/@jamesheathers/the-unbearable-heaviness-of-text-recycling-12389fe9850d
This is one of the most important decisions in an EFA (Thompson, 2004; Warne & Larsen, 2014), and these decisions can make g artificially easier or harder to identify.
Deciding the number of factors to retain can be extremely subjective. But that was why it was so important to pre-register our work. We wanted to choose methods for making this decision before seeing the data so that no one could accuse of us of trying to monkey with the data until we got the results we wanted.
For the sake of transparency, we find it important to explicitly state deviations from our preregistration protocol. First, in our preregistration, we stated that we would search for (cognitive OR intelligence) AND the name of a continent or population. However, searching for a continent was not feasible in finding data sets. We also had difficulty generating a list of population groups (e.g., ethnic groups, tribal groups) that would be useful for our search procedures.
This was my second time I pre-registered the study and the first time my student co-author had. We are still getting the hang of it.
Alternatively, one could postulate that a general cognitive ability is a Western trait but not a universal trait among humans, but this would require an evolutionary model where this general ability evolved several times independently throughout the mammalian clade, including separately in the ancestors of Europeans after they migrated out of Africa and separated from other human groups. Such a model requires (a) a great deal of convergent evolution to occur across species occupying widely divergent environmental niches and (b) an incredibly rapid development of a general cognitive ability while the ancestors of Europeans were under extremely strong selection pressures that other humans did not experience (but other mammal species or their ancestors would have experienced at other times). We find the more parsimonious model of an evolutionary origin of the general cognitive ability in the early stages of mammalian development to be the more plausible one, and thus we believe that it is reasonable to expect a general cognitive ability to be a universal human trait.
It was this reasoning that led to the decision to conduct this study. There is mounting evidence that g exists in other mammalian species, and it definitely exists in Western cultures. It seemed really unlikely that it would not exist in non-Western groups. But I couldn't find any data about the issue. So, time to do a study!
Researchers studying the cognitive abilities of animals have identified a general factor in cognitive data from many mammal species
Also, cats:
Warren, J. M. (1961). Individual differences in discrimination learning by cats. The Journal of Genetic Psychology, 98, 89-93. doi:10.1080/00221325.1961.10534356
The same logic that researchers use to argue that a folk belief regarding intelligence provides evidence of the nature of intelligence could also be used to argue that widespread cultural beliefs in elves, goblins, or angels provide evidence of the existence of supernatural beings. Survey methodology regarding layman beliefs cannot substitute for collecting data on individuals’ performance on mental tasks.
I had fun writing this passage.
Investigating cultural beliefs about intelligence may be mildly interesting from an anthropological perspective, but it sheds little light on the nature of intelligence. One undiscussed methodological problem in many studies of cultural perspectives on intelligence is the reliance on surveys of laymen to determine what people in a given culture believe about intelligence. This methodology says little about the actual nature of intelligence.
After the manuscript was accepted for publication and the proofs turned in, I re-discovered the following from Gottfredson (2003, p. 362): ". . . lay beliefs are interesting but their value for scientific theories of intelligence is limited to hypothesis generation. Even if the claim were true, then, it would provide no evidence for the truth of any intelligence theory . . ."
Gottfredson, L. S. (2003). Dissecting practical intelligence theory: Its claims and evidence. Intelligence, 31, 343-397. doi:10.1016/S0160-2896(02)00085-5
Panga Munthu test of intelligence
To me, this is the way to create tests of intelligence for non-Western cultures: find skills and manifestations of intelligence that are culturally appropriate for a group of examinees and use those skills to tap g. Cross-cultural testing would require identifying skills that are valued or developed in both cultures.
Berry (1986)
John W. Berry is a cross-cultural psychologist whose work stretches back over 50 years. He takes the position (e.g., Berry, 1986) that definitions of intelligence are culturally-specific and are bound up with the skills cultures encourage and that the environment requires people to develop. Therefore, he does not see Western definitions as applying to most groups.
After this study, my position is more nuanced approach. I agree with Berry that the manifestations of intelligence can vary from culture to culture, but that underneath these surface features is g in all humans.
We believe that members of the public likely learn some inaccurate information about intelligence in their psychology courses. The good news about this implication is that reducing the public’s mistaken beliefs about intelligence will not take a massive public education campaign or public relations blitz. Instead, improving the public’s understanding about intelligence starts in psychology’s own backyard with improving the content of undergraduate courses and textbooks.
To me, this is the "take home" message of the article. I hope psychology educators do more to improve the accuracy of their lessons about intelligence. I also hope more programs add a course on the topic to their curriculum.
This means that it is actually easier to measure intelligence than many other psychological constructs. Indeed, some individuals trying to measure other constructs have inadvertently created intelligence tests
When I learned this, it blew my mind.
Lewontin’s fallacy
For the article to name this fallacy and thoroughly debunk it, see Edwards, A. W. F. (2003). Human genetic diversity: Lewontin's fallacy. BioEssays, 25, 798-801. doi:10.1002/bies.10315
many psychologists simply accept an operational definition of intelligence by spelling out the procedures they use to measure it. . . . Thus, by selecting items for an intelligence test, a psychologist is saying in a direct way, “This is what I mean by intelligence.” A test that measures memory, reasoning, and verbal fluency offers a very different definition of intelligence than one that measures strength of grip, shoe size, hunting skills, or the person’s best Candy Crush mobile game score. (p. 290)
Ironically, there is research showing that video game performance is positively correlated with intelligence test scores (e.g., Angeles Quiroga et al., 2015; Foroughi, Serraino, Parasuraman, & Boehm-Davis, 2016).
Not every inaccurate statement in the textbooks was as silly as this one. Readers would benefit from browsing Supplemental File 2, which
Minnesota Transracial Adoption Study
This is a study begun in the 1970s of African American, interracial, and other minority group children who had been adopted by White families in Minnesota. The 1976 results indicated large IQ boosts (about 12 points) for adopted African American children at age 6, compared to the average IQ for African Americans in general. However, the 1992 report shows that the advantage had faded to about 6 points when the children were aged 17 years. Generally, intelligence experts see this landmark study as supporting both "nature" and "nurture."
the Stanford-Binet intelligence test
Although the Stanford-Binet is historically important, the Wechsler family of intelligence tests have been more popular since the 1970s.
The introductory psychology textbook is difficult to produce with uniform accuracy, as authors have only a limited area of expertise, yet must write chapters that discuss the entire breadth of psychology.
My heart goes out to anyone trying to write an introductory psychology textbook. It's impossible to be an expert on every area of psychology, so some weaknesses are inevitable. These authors are trying their best.
Mayson C. Astle
Very proud to have a student as a co-author on this article.
Some readers will also be surprised to find that The Bell Curve is not as controversial as its reputation would lead one to believe (and most of the book is not about race at all).
I wrote this sentence. Two coauthors, three peer reviewers, and an editor all read it multiple times. No one ever asked for it to be changed.
Gardner’s multiple intelligences
I have a Twitter moment that analyzes Gardner's book "Frames of Mind" and shows why this theory is poorly supported by empirical data. https://twitter.com/i/moments/1064036271847161857
Most frequently this appeared in the form of a tacit acknowledgment that IQ test scores correlate with academic success, followed by a quick denial that the scores are important for anything else in life
We discussed this point with a reporter for an article that made the front page of the local newspaper: https://www.heraldextra.com/news/local/education/college/uvu/uvu-researchers-found-incorrect-info-in-percent-of-psychology-textbooks/article_6a84fddd-3dba-59bb-9a7e-ed957619ff20.html
this study highlights the mismatch between scholarly consensus on intelligence and the beliefs of the general public
Christian Jarrett of the The British Psychological Society found this as the main message of the article. Read his blog post at https://digest.bps.org.uk/2018/03/08/best-selling-introductory-psychology-books-give-a-misleading-view-of-intelligence/
In recent years, the evidence that genes are at least a partial influence of every human behavior and psychological trait has mounted so quickly that the early 21st century may be the dawn of a behavioral genetics revolution in psychology. Such a revolution may be as important—or more important—for psychology than the cognitive revolution was in the mid-20th century.
This is one of the most important quotes in the article. Check with me in 30 years to see if my coauthors and I are correct.
Judged solely by the number of factually inaccurate statements, the textbooks we examined were mostly accurate.
A blog post by James Thompson (psychology professor emeritus at University College London) has a much more acerbic response to the study than this. See his blog post for a contrasting viewpoint: http://www.unz.com/jthompson/fear-and-loathing-in-psychology/
We found that 79.3% of textbooks contained inaccurate statements and 79.3% had logical fallacies in their sections about intelligence.
"Psychology Today" article on this study: https://www.psychologytoday.com/us/blog/brainstorm/201803/psychology-textbooks-contain-inaccuracies-intelligence
Neisser et al.’s (1996) article is the official report of an APA committee to produce a summary of intelligence research.
Another classic article, though some of its findings (e.g., on stereotype threat) are starting to become outdated.
Gottfredson’s (1997a) mainstream statement on intelligence
This article is a classic, and it is required reading in my undergraduate human intelligence course. If you only have time to read 1 article about intelligence, this should be it.
Warne, R. T., Astle, M. C., & Hill, J. C. (2018). What do undergraduates learn about human intelligence? An analysis of introductory psychology textbooks. Archives of Scientific Psychology, 6(1), 32-50. http://dx.doi.org/10.1037/arc0000038
Freedom Radio interview about the article and related issues: https://www.youtube.com/watch?v=EQO8I-j3Eac
What Do Undergraduates Learn About Human Intelligence? An Analysis of Introductory Psychology Textbooks
This was the 8th most downloaded article that APA published in 2018. Here is the "Monitor on Psychology" article about that: https://www.apa.org/monitor/2018/12/top-journal-articles.aspx
ABSTRACT
Non-scientific summary available from Kudos: https://www.growkudos.com/publications/10.1037%25252Farc0000038/reader
Figure 3.
I can't believe we got away with putting this image into a scholarly article.