On 2016 Jun 22, Christopher Tench commented:

Thanks for the update Xin. The false positives I refer to are in relation to that expected from the stated method. For any method there needs to be a decision made, before the experiment, about what risk of false positive is acceptable. This may be arbitrary, as you say. Nevertheless, the results only meet the accepted risk when the method is implemented correctly. The problem with the implementation prior to 2.3.6 is that it did not control the FWE. As such there was no way to know whether there were any significant results at all until, as you have done, the appropriate checks and updates were performed.

Chris

On 2016 Jun 21, Xin Di commented:

Hi, Chris,

Thank you for your interest in our paper.

The bug you referred to was announced on April 26, 2016 (2.3.6 http://brainmap.org/ale/readme.html). Our data analysis was performed, and our manuscript was submitted before this date. I could not see any possibility that we could use the 2.3.6 version in our paper.

As described in the Brainmap forum, this bug makes cluster-level threshold more lenient. We re-analyzed our data using GingerALE version 2.3.6, and confirmed that some small clusters reported in our analysis were no longer significant at the same threshold of cluster-level p < 0.05. But large clusters for each of the analyses are still significant. The idea of our paper is that there are consistent task modulated connectivity with the amygdala, and different tasks may modulate amygdala connectivity with different brain regions. Our conclusion will not be affected if we used version 2.3.6, because it is not based on any single clusters.

Lastly, your comment that older versions of GingerALE have "a bug that produces false positive results" is only partially correct. Indeed, all statistical methods produce false positive results, not to mention the so-called type II error. The bugged version is more likely to produce false positive results. It doesn't mean that all the results are false positive. The statistical threshold is arbitrary, anyway. Why do we use p < 0.05, but not p < 0.03 or p < 0.080808? My point is, when drawing conclusions from data, we need to consider the pattern of results, but not a specific result from arbitrarily picked threshold.

Best, Xin

On 2016 Jun 13, Christopher Tench commented:

The version of GingerALE used in this paper has a bug that produces false positive results. The bug has recently been fixed in version 2.3.6.

On 2016 Jun 13, Christopher Tench commented:

The version of GingerALE used in this paper has a bug that produces false positive results. The bug has recently been fixed in version 2.3.6.