On 2015 Jun 22, Eileen Daly commented:

Dear Professor Bishop,

Thank you so much for inviting us, through PubMed Commons, to reply to your comments on our Brain paper Response inhibition and serotonin in autism: a functional MRI study using acute tryptophan depletion. Your comments highlighted the differences between our paper and that of Chantiluke et al.’s 2015 paper Inverse fluoxetine effects on inhibitory brain activation in non-comorbid boys with ADHD and with ASD.

From your comments, I sense your main issue is that our baseline data appears to differ from that of Chantiluke. This is of course a very interesting issue.

There were many differences between the two studies that should be taken into account when comparing the findings.

• The sample of Daly et al. was adults (mean age = 31) while that of Chantiluke et al. was pediatric (mean age = 15).

• It is not correct that both studies used a Go/No-Go (GNG) task. The tasks were not the same. The study of Chantiluke compared successful stop with failed stop trials, whereas the study of Daly compared successful No-Go with oddball trials. While both studies controlled for the oddball effect, the task demands are different - the stop task is more challenging and has a lower load on selective attention than the GNG task.

• The location of the right inferior frontal (R IFC) activation. The location of the R IFC underactivation in the study of Daly is in a more rostral and ventral location of ventrolateral prefrontal cortex at a z score between -12 and + 18 comprising BA 47, 10 and 46. It is hence a more ventral and rostral IFC region. In contrast the location of the IFC in the paper of Chantiluke only appears at + 30 to +40 and is in BA 44/9, - i.e. a far more dorsal and somewhat more caudal location. Therefore the two regions are not identical and the findings are consequently not inconsistent. Our most recent and yet unpublished (Lukito S, Carlisi C, Radua J, Simonoff E, Rubia K) meta-analysis of 13 inhibition fMRI datasets in 193 ASD children and adults shows in fact that the more dorsal location of IFC in BA 44/9 at a z-score of + 20 (comparable to IFC location in Chantiluke’s paper) is overactivated in ASD while a more ventral location in BA 47/46 at a z-score of 0 (comparable to the IFC location in Daly’s paper) is underactivated in ASD. The findings of the two papers are also in line with the findings of Kana et al., who found a more ventral location of IFC (BA 45) at + 10 to be underactivated in ASD, which corresponds to the IFC location of the study of our findings in Daly et al.

• It is very difficult to compare the two baseline conditions across studies. First, Chanktiluke’s baseline is a true “placebo” condition, in that an “inactive’’ substance was utilized, as the placebo was peppermint water. In our study, however, the baseline condition is actually a “sham” procedure with a drink containing many “active” (i.e. crosses blood-brain barrier) substances, which were amino acids in our case. Thus, peppermint water has no physiological effect on brain functioning while our amino acid drink includes molecules that will exert a direct effect upon the baseline. This important issue may hence explain why differences in baseline conditions were reported.

However, we would argue that the most consistent explanation for this apparent inconsistency and that is also in line with our recent meta-analysis of fMRI studies of inhibition in ASD is that there are two different IFC regions that are functionally abnormal in ASD during inhibition tasks. A more ventral and rostral region in BA 47/46 that is underactivated and a more ventral and caudal region in BA 44/9 that is overactivated. Serotonin modulation appears to normalize both deviant activations, in that it enhances the underactivated BA 47/46 cluster and it reduces the overactivated BA 44/9 cluster moving both activation patterns towards normality.

Last, we wholehearted agree that the reproducibility in neuroscience and pharmacological intervention studies are critical. We also fully agree that this is an initial proof-of-concept study (albeit one that involved over 60 separate imaging experiments) that requires replication. However, our paper fully acknowledged this issue and used state-of-the art statistical approaches for analysing small samples (e.g. non-parametric statistics). Furthermore, we correct for multiple comparisons using standard methods (e.g. clusterwise for fMRI data and FDR for correlations). We were very pleased to observe that the finding of BA 46/47 underactivation was replicated in our meta-analysis in a much larger sample of 193 ASD patients and are hence confident in the finding.

Again, we thank you for your invitation to reply to your comment and value your input.

Sincerely,

Dr Eileen Daly

On 2015 May 31, Dorothy V M Bishop commented:

I was searching for the original source of a claim about autism by Declan Murphy: "we have found this brain abnormality, and we’ve been able to show we can reverse it" http://alumni.kcl.ac.uk/pioneers-appeal-professor-declan-murphy, and I came across this paper. I'm not sure if it the source of the claims: there is another paper on a closely similar topic with overlapping authors by Chantiluke et al (2015).

I am puzzled, though, by the fact that these two papers seem to have inconsistent findings with regard to ASD, yet do not comment on this or cross-reference one another. Indeed, in the current paper, it is stated that: "to our knowledge, no one has investigated the modulatory effect of serotonin on inhibitory neural activity in individuals with ASD" (p 2601); the Chantiluke et al paper was not published at the time that Daly et al was submitted, but it was submitted two weeks later.

Both papers look at behavioural and brain responses to a Go-NoGo task in individuals with ASD, with a child sample in Chantiluke et al, and adults in Daly et al. Serotonin levels are modified by fluoxetine administration in Chantiluke et al, and by tryptophan depletion in Daly et al. There are some procedural differences in the Go-NoGo task, but my impression is that these were not so great that one would expect to see a major effect on results. Yet quite apart from any impact of serotonin manipulations, the studies find different results in the placebo conditions. Daly et al find that "after sham, subjects with autism relative to control subjects, showed reduced activation in right inferior frontal cortex". In contrast, Chantiluke et al reported "Under placebo, relative to controls, ASD boys showed overactivation in left and right inferior frontal cortex (IFC)." (p. 2071).

In the introduction to these papers, both cite prior work on inferior frontal cortex (IFC) and inhibition in autism. In particular, they make predictions based on another study by this group, by Schmitz et al, which also used fMRI to compare ASD and control groups during a Go-NoGo task. Schmitz et al reported significant increase in left middle/inferior (Brodmann area 10/46) and orbitofrontal gyrus during correct inhibition of NoGo trials, and no significant areas of decreased activation (p 11-12). Yet, Daly et al hypothesise that in the placebo conditions "subjects with ASD compared to controls would show decreased activation in right inferior frontal and increased activation in left inferior frontal cortices".

The importance of reproducibility of research findings in neuroscience is receiving increased attention (Pernet & Poline, 2015), with growing recognition that there is a major problem for neuroimaging studies with small samples (Button et al., 2013), especially when, as in this case, multiway Anovas and correlations are computed without correction for multiple comparisons. Pharmacological interventions that 'normalise' brain abnormalities are potentially of huge interest, but they are unlikely to reflect more than error of measurement and regression to the mean if baseline differences are not reproducible.

It may be that there is some simple explanation for these inconsistencies; it would be good to hear the authors' views on this.

References Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafo, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376. doi: 10.1038/nrn3475

Chantiluke, K., Barrett, N., Giampietro, V., Santosh, P., Brammer, M., Simmons, A., Murphy, D., & Rubia, K. (2015). Inverse fluoxetine effects on inhibitory brain activation in non-comorbid boys with ADHD and with ASD. Psychopharmacology, 232(12), 2071-2082.

Pernet, C., & Poline, J.-B. (2015). Improving functional magnetic resonance imaging reproducibility. GigaScience, 4(15). doi: 10.1186/s13742-015-0055-8

Schmitz, N., Rubia, K., Daly, E., Smith, A., Williams, S., & Murphy, D. G. M. (2006). Neural Correlates of Executive Function in Autistic Spectrum Disorders. Biological Psychiatry, 59(1), 7-16

On 2015 May 31, Dorothy V M Bishop commented:

I was searching for the original source of a claim about autism by Declan Murphy: "we have found this brain abnormality, and we’ve been able to show we can reverse it" http://alumni.kcl.ac.uk/pioneers-appeal-professor-declan-murphy, and I came across this paper. I'm not sure if it the source of the claims: there is another paper on a closely similar topic with overlapping authors by Chantiluke et al (2015).

I am puzzled, though, by the fact that these two papers seem to have inconsistent findings with regard to ASD, yet do not comment on this or cross-reference one another. Indeed, in the current paper, it is stated that: "to our knowledge, no one has investigated the modulatory effect of serotonin on inhibitory neural activity in individuals with ASD" (p 2601); the Chantiluke et al paper was not published at the time that Daly et al was submitted, but it was submitted two weeks later.

Both papers look at behavioural and brain responses to a Go-NoGo task in individuals with ASD, with a child sample in Chantiluke et al, and adults in Daly et al. Serotonin levels are modified by fluoxetine administration in Chantiluke et al, and by tryptophan depletion in Daly et al. There are some procedural differences in the Go-NoGo task, but my impression is that these were not so great that one would expect to see a major effect on results. Yet quite apart from any impact of serotonin manipulations, the studies find different results in the placebo conditions. Daly et al find that "after sham, subjects with autism relative to control subjects, showed reduced activation in right inferior frontal cortex". In contrast, Chantiluke et al reported "Under placebo, relative to controls, ASD boys showed overactivation in left and right inferior frontal cortex (IFC)." (p. 2071).

In the introduction to these papers, both cite prior work on inferior frontal cortex (IFC) and inhibition in autism. In particular, they make predictions based on another study by this group, by Schmitz et al, which also used fMRI to compare ASD and control groups during a Go-NoGo task. Schmitz et al reported significant increase in left middle/inferior (Brodmann area 10/46) and orbitofrontal gyrus during correct inhibition of NoGo trials, and no significant areas of decreased activation (p 11-12). Yet, Daly et al hypothesise that in the placebo conditions "subjects with ASD compared to controls would show decreased activation in right inferior frontal and increased activation in left inferior frontal cortices".

The importance of reproducibility of research findings in neuroscience is receiving increased attention (Pernet & Poline, 2015), with growing recognition that there is a major problem for neuroimaging studies with small samples (Button et al., 2013), especially when, as in this case, multiway Anovas and correlations are computed without correction for multiple comparisons. Pharmacological interventions that 'normalise' brain abnormalities are potentially of huge interest, but they are unlikely to reflect more than error of measurement and regression to the mean if baseline differences are not reproducible.

It may be that there is some simple explanation for these inconsistencies; it would be good to hear the authors' views on this.

References Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafo, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376. doi: 10.1038/nrn3475

Chantiluke, K., Barrett, N., Giampietro, V., Santosh, P., Brammer, M., Simmons, A., Murphy, D., & Rubia, K. (2015). Inverse fluoxetine effects on inhibitory brain activation in non-comorbid boys with ADHD and with ASD. Psychopharmacology, 232(12), 2071-2082.

Pernet, C., & Poline, J.-B. (2015). Improving functional magnetic resonance imaging reproducibility. GigaScience, 4(15). doi: 10.1186/s13742-015-0055-8

Schmitz, N., Rubia, K., Daly, E., Smith, A., Williams, S., & Murphy, D. G. M. (2006). Neural Correlates of Executive Function in Autistic Spectrum Disorders. Biological Psychiatry, 59(1), 7-16