On 2022-05-20 03:55:55, user Jake Gratten wrote:
Response to Morton et al. (2022): model mis-specification criticism overlooks sensitivity analyses and orthogonal analyses
The core criticism of our study (Yap et al., 2021) made by Morton et al. was that the linear mixed model (LMM) framework we employed includes a questionable biological assumption – that diet and the microbiome are independent. They correctly note that diet is known to influence the microbiome (David et al., 2014; Rothschild et al., 2018), and thus, as these factors are inter-related, our model may be prone to biased inference. We acknowledge these points in relation to the specific LMM (see below) on which the critique by Morton et al. is focused. However, we respectfully disagree with their conclusion that this issue invalidates the findings reported in our paper, because their critique (1) incorrectly asserts that this result formed the basis of our conclusions, and (2) it overlooks several key analyses, including extensive sensitivity analyses that were specifically performed to test this (and other) assumptions.
Morton and colleagues focus on a single LMM analysis of ASD in their critique, in which we adjusted for sex, age and diet, the latter by fitting the top three principal components from PCA of the centre log ratio (clr)-transformed percent energy variables from the Australian Eating Survey (AES), a validated food frequency questionnaire. In this analysis, we found that 0% of the variance in ASD diagnosis was associated with the microbiome, irrespective of the microbiome features used to construct the correlation matrix describing the relationships between random effects (e.g., common species, rare species, common genes, rare genes) (Yap et al., 2021). As diet is correlated with the microbiome, it is possible that adjusting for diet in this analysis has removed variance in ASD diagnosis that may be attributable to the microbiome. In their critique, the authors present simulations purporting to show that this issue could lead to failure to detect even very large proportions of variance (in their example 83%) (Morton, Donovan, & Taroncher-Oldenburg, 2022).
Unfortunately, they fail to mention that we also performed a LMM analysis of ASD in which we did not adjust for diet (or sex or age). If there was an effect of the microbiome on ASD that had previously been removed by adjusting for diet, then this should now be “revealed” (i.e., captured by the microbiome random effect). However, we found precisely the same result as in our original analysis: that is, 0% of the variance in ASD diagnosis is associated with the microbiome (Yap et al., 2021). Based upon this analysis of the available data we believe it is unlikely that our conclusions have been biased by model mis-specification.
The authors also do not acknowledge that we performed LMM analyses of traits other than ASD, and whereas there was negligible signal for ASD, IQ and sleep problems, we found large and significant associations of the microbiome with age, sex and stool consistency. Our results for age (i.e., ~30% of the variance associated with common microbiome species) are particularly notable because they recapitulate the findings reported in a large (independent) sample of >30K adult stool metagenomes (Rothschild et al., 2020). Our LMM results for age, sex and stool consistency were also largely unaffected by adjusting for diet (Yap et al., 2021). These analyses, which were specifically included for the purpose of benchmarking the findings for ASD, provide further evidence that our methods are not prone to under-estimating the proportion of trait variance associated with the microbiome.
It is also relevant to highlight that the directionality of the causal graphs presented by Morton et al. in Figure 1 of their article (i.e., a causal effect of both the microbiome and diet on the host phenotype) are problematic, since the variance component estimates from these models might reflect cause or consequence of the focal trait. This is because microbiome taxonomic proportions change, unlike genotypes used in analogous LMM methods for estimating heritability (which are present at birth and therefore representative of causality). To demonstrate this, take as an example our analysis in which age was the dependent variable and microbiome measures were fitted as random effects (allowing capture of their interdependence). We find roughly 30% of the variance in age is associated with common microbiome species. Clearly, the way to interpret this result is that age is causal for the variance in the microbiome, not the other way round. It is equally possible that ASD influences diet and in turn the microbiome, as opposed to the opposite view espoused by Morton et al. Indeed, the wording used in their critique (i.e., “A more accurate model would have assumed an architecture that explicitly incorporates the direct influence of diet on the ASD phenotype as well as an indirect influence of diet on the ASD phenotype via the microbiome”) appears not to recognise this possibility.
Looking beyond the LMM analyses in our paper, Morton and colleagues also did not consider several other key sets of analyses on which are conclusions are based, including differential abundance testing using ANCOM (Analysis of Composition of Microbiomes) and extensive linear model analyses. In our ANCOM analysis of ASD, we find a single robustly associated species (Romboutsia timonensis) when adjusting for sex, age and dietary PCs, but this same species remains the only significant finding in analyses without covariates (Yap et al., 2021). This is entirely consistent with our LMM model findings but is not what would be expected if the microbiome was associated with a high proportion of variance in ASD diagnosis. Indeed, irrespective of how the data are analysed (e.g., sibling pairs only, excluding siblings, excluding children with recent exposure to antibiotics, and others), we find negligible evidence for association of individual species with ASD (other than R. timonensis), and no support whatsoever for taxa previously reported to be associated with ASD.
In our linear model analyses, we show that quantitative measures of the autism spectrum, including both psychometric measures (e.g., ADOS-2/G Restricted and Repetitive Behaviour (RRB) calibrated severity scores) and polygenic scores were associated with reduced dietary diversity (Yap et al., 2021). The most parsimonious interpretation of these findings is that RRBs, which are one of the core diagnostic signs of ASD, manifest in the form of more selective dietary preferences. Polygenic scores, as an immutable component of propensity to ASD-associated traits, are an important and novel aspect of our analysis, given they facilitate preliminary causal inference (noting that we were careful to avoid strong statements about causality in our paper). In contrast, other cross-sectional autism microbiome studies – whose results have been prioritised by Morton et al. – have not exploited genetic predictors for autism-related traits and so cannot distinguish between cause and consequence.
Overall, using a variety of orthogonal analytical approaches, we find a strong and consistent signal that ASD (and autistic traits) is associated with reduced dietary diversity, and that diet in turn is associated with the microbiome (Yap et al., 2021). These results are consistent with existing evidence for dietary effects on the microbiome (David et al., 2014; Rothschild et al., 2018) – as pointed out by Morton et al. – and with prior evidence (backed by clinical and lived experience) for an association of autism with diet (Berding & Donovan, 2018). We find no direct association of ASD with the microbiome, a result to which Morton and colleagues express surprise, their argument being that if ASD is associated with diet and diet influences the microbiome, then how can there be no direct ASD-microbiome association? The answer is simply that we have a finite sample, and the effect sizes are subtle. We expect that in a larger sample we might observe a direct association, but also stronger evidence that this is due to changes in diet that are related to autistic traits. This is a considerably more intuitive and parsimonious explanation for associations of the microbiome with ASD than the idea that the microbiome contributes to autistic traits, not least because there is strong evidence that ASD is a neuro-developmental condition, and expression of established ASD genes is enriched prenatally (Satterstrom et al., 2020). In this context, it is worth emphasising that the high estimated heritability of ASD (70-80%) (Bai et al., 2019) leaves relatively little room for other putative etiological causal factors (e.g., maternal immune activation). This is especially true given de novo mutations that are known to be important in ASD (Sanders et al., 2015; Sanders et al., 2012; Satterstrom et al., 2020) largely do not contribute to heritability estimates (i.e., because they are not shared by relatives) and so must consume an additional proportion of the remaining 20-30% of variance.
Morton et al.’s criticism of our study comes despite it being the largest (and therefore most statistically well-powered) to date. Our study also has the dual benefits of matching data on diet and other confounders, which are lacking in many prior studies, and deep metagenomic sequencing, compared to inferior 16S technology in most published ASD microbiome papers. We note that ours is not the first study to report negligible association of the microbiome with ASD (Gondalia et al., 2012; Son et al., 2015). We also point to a recent review in Cell on microbiome studies in animal models (including for autism) highlighting the implausibility of the high proportion of positive findings, asserting that the field suffers from publication bias (Walter, Armet, Finlay, & Shanahan, 2020). That said, we acknowledge that our study has limitations, reflecting difficulties of collecting idealised data sets. Prospective studies collecting faecal samples from infants prior to autism diagnosis are needed to further advance the field, but these are challenging both logistically and because sample size is limited by the population prevalence of ASD (~1%).
To sum up, we thank Morton et al. for their comments in relation to one specific analysis in our paper. This provides us with the opportunity to clarify the detailed analyses that we performed to reach our conclusions. Unfortunately, the critique from Morton et al. (based solely on simulations) overlooks most of our results, including sensitivity analyses that directly address their criticism. The authors suggest that our data should be re-analysed. We note that our data are available by application to the Australian Autism Biobank which allows other researchers to provide objective empirical evaluation. We are committed to transparent research and provide extensive supplementary materials and publicly available code and hope others in the research community will build upon our work.
Chloe X. Yap, Peter M. Visscher, Naomi R. Wray and Jacob Gratten <br />
(On behalf of all authors)
References<br />
Bai, D., Yip, B. H. K., Windham, G. C., Sourander, A., Francis, R., Yoffe, R., . . . Sandin, S. (2019). Association of Genetic and Environmental Factors With Autism in a 5-Country Cohort. JAMA Psychiatry, 76(10), 1035-1043. doi:10.1001/jamapsychiatry.2019.1411
Berding, K., & Donovan, S. M. (2018). Diet Can Impact Microbiota Composition in Children With Autism Spectrum Disorder. Front Neurosci, 12, 515. doi:10.3389/fnins.2018.00515
David, L. A., Maurice, C. F., Carmody, R. N., Gootenberg, D. B., Button, J. E., Wolfe, B. E., . . . Turnbaugh, P. J. (2014). Diet rapidly and reproducibly alters the human gut microbiome. Nature, 505(7484), 559-563. doi:10.1038/nature12820
Gondalia, S. V., Palombo, E. A., Knowles, S. R., Cox, S. B., Meyer, D., & Austin, D. W. (2012). Molecular characterisation of gastrointestinal microbiota of children with autism (with and without gastrointestinal dysfunction) and their neurotypical siblings. Autism Res, 5(6), 419-427. doi:10.1002/aur.1253
Morton, J. T., Donovan, S. M., & Taroncher-Oldenburg, G. (2022). Decoupling diet from microbiome dynamics results in model mis-specification that implicitly annuls potential associations between the microbiome and disease phenotypes—ruling out any role of the microbiome in autism (Yap et al. 2021) likely a premature conclusion. biorxiv. doi:https://doi.org/10.1101/202...
Rothschild, D., Leviatan, S., Hanemann, A., Cohen, Y., Weissbrod, O., & Segal, E. (2020). An atlas of robust microbiome associations with phenotypic traits based on large-scale cohorts from two continents. biorxiv. doi:https://doi.org/10.1101/202...
Rothschild, D., Weissbrod, O., Barkan, E., Kurilshikov, A., Korem, T., Zeevi, D., . . . Segal, E. (2018). Environment dominates over host genetics in shaping human gut microbiota. Nature, 555(7695), 210-215. doi:10.1038/nature25973
Sanders, S. J., He, X., Willsey, A. J., Ercan-Sencicek, A. G., Samocha, K. E., Cicek, A. E., . . . State, M. W. (2015). Insights into Autism Spectrum Disorder Genomic Architecture and Biology from 71 Risk Loci. Neuron, 87(6), 1215-1233. doi:10.1016/j.neuron.2015.09.016
Sanders, S. J., Murtha, M. T., Gupta, A. R., Murdoch, J. D., Raubeson, M. J., Willsey, A. J., . . . State, M. W. (2012). De novo mutations revealed by whole-exome sequencing are strongly associated with autism. Nature, 485(7397), 237-241. doi:10.1038/nature10945
Satterstrom, F. K., Kosmicki, J. A., Wang, J., Breen, M. S., De Rubeis, S., An, J. Y., . . . Buxbaum, J. D. (2020). Large-Scale Exome Sequencing Study Implicates Both Developmental and Functional Changes in the Neurobiology of Autism. Cell, 180(3), 568-584 e523. doi:10.1016/j.cell.2019.12.036
Son, J. S., Zheng, L. J., Rowehl, L. M., Tian, X., Zhang, Y., Zhu, W., . . . Li, E. (2015). Comparison of Fecal Microbiota in Children with Autism Spectrum Disorders and Neurotypical Siblings in the Simons Simplex Collection. PLoS ONE, 10(10), e0137725. doi:10.1371/journal.pone.0137725
Walter, J., Armet, A. M., Finlay, B. B., & Shanahan, F. (2020). Establishing or Exaggerating Causality for the Gut Microbiome: Lessons from Human Microbiota-Associated Rodents. Cell, 180(2), 221-232. doi:10.1016/j.cell.2019.12.025
Yap, C. X., Henders, A. K., Alvares, G. A., Wood, D. L. A., Krause, L., Tyson, G. W., . . . Gratten, J. (2021). Autism-related dietary preferences mediate autism-gut microbiome associations. Cell, 184(24), 5916-5931 e5917. doi:10.1016/j.cell.2021.10.015