On 2014 Dec 02, Mikkel Wallentin commented:

On the behavioral effects, the authors write: "Compared with the low-flavanol intervention, subjects who received the high-flavanol intervention showed a mean improved cognitive performance of 630 ms." However, visual inspection of the supplementary figure 3 clearly shows that the 630 ms difference at follow up is due to a modest improvement of approx. 200 ms between baseline and follow up and a much more pronounced DECLINE in the control group (approx. 400ms). Needless to say, such a difference cannot be interpreted as "a mean improved cognitive performance of 630 ms", if it can be interpreted at all.

On 2014 Oct 27, David Colquhoun commented:

For all the reasons given by Hilda Bastian (and a few more, like P = 0.04 provides lousy evidence) it astonishes me that this study should have been trumpeted as though it represented a great advance. That's the responsibility of Nature Neuroscience (and, ultimately, of the authors).

I wonder whether what happens is as follows. Authors do big fMRI study. Glamour journal refuses to publish without functional information. Authors tag on a small human study. Paper gets published. Hyped up press releases issued that refer mostly to the add on. Journal and authors are happy. But science is not advanced.

I certainly got this impression in another recent fMRI paper in Science. Brain stimulation was claimed to improve memory (P = 0.043)

I guess these examples are quite encouraging for those who think that expensive glamour journals have had their day. Open access and open comments are the way forward.

On 2014 Oct 27, Hilda Bastian commented:

This report of a very small, short-term trial in healthy adults does not meet the CONSORT standards for trial reporting in several key respects. It does not provide sufficient data on the cognitive outcomes assessed, nor an adequate flow chart of outcomes (despite considerable attrition). There is also very little detail provided in the record of this trial at ClinicalTrials.gov.

The abstract does not make it clear that this is a dietary supplement and exercise trial (partially funded by a manufacturer). There were apparently two cognitive outcome measures on a ModBent task (an adapted test not elsewhere validated): immediate matching and delayed retention. Both relate to very specific functions, not an overall rating of cognitive abilities.

No effect was found for the exercise component in the trial, and out of the two cognitive measures, some effect was found for one, but not the other. That this is a chance finding surely can't be ruled out.

This report describes low vs high supplement groups. The study in ClinicalTrials.gov for the trial number they provide, however, was for a supplement and a placebo comparator.

Despite the major limitations of this single trial to address the question, the "Newsroom" report for the trial claims that it shows that "dietary flavanols reverse age-related memory decline."

It's good to see claims about dietary supplements tested. However, the results here rely on a chain of yet-to-be-validated assumptions that are still weakly supported at each point. In my opinion, the immodest title of this paper is not supported by its contents.

On 2014 Oct 27, Hilda Bastian commented:

This report of a very small, short-term trial in healthy adults does not meet the CONSORT standards for trial reporting in several key respects. It does not provide sufficient data on the cognitive outcomes assessed, nor an adequate flow chart of outcomes (despite considerable attrition). There is also very little detail provided in the record of this trial at ClinicalTrials.gov.

The abstract does not make it clear that this is a dietary supplement and exercise trial (partially funded by a manufacturer). There were apparently two cognitive outcome measures on a ModBent task (an adapted test not elsewhere validated): immediate matching and delayed retention. Both relate to very specific functions, not an overall rating of cognitive abilities.

No effect was found for the exercise component in the trial, and out of the two cognitive measures, some effect was found for one, but not the other. That this is a chance finding surely can't be ruled out.

This report describes low vs high supplement groups. The study in ClinicalTrials.gov for the trial number they provide, however, was for a supplement and a placebo comparator.

Despite the major limitations of this single trial to address the question, the "Newsroom" report for the trial claims that it shows that "dietary flavanols reverse age-related memory decline."

It's good to see claims about dietary supplements tested. However, the results here rely on a chain of yet-to-be-validated assumptions that are still weakly supported at each point. In my opinion, the immodest title of this paper is not supported by its contents.

On 2014 Oct 27, David Colquhoun commented:

For all the reasons given by Hilda Bastian (and a few more, like P = 0.04 provides lousy evidence) it astonishes me that this study should have been trumpeted as though it represented a great advance. That's the responsibility of Nature Neuroscience (and, ultimately, of the authors).

I wonder whether what happens is as follows. Authors do big fMRI study. Glamour journal refuses to publish without functional information. Authors tag on a small human study. Paper gets published. Hyped up press releases issued that refer mostly to the add on. Journal and authors are happy. But science is not advanced.

I certainly got this impression in another recent fMRI paper in Science. Brain stimulation was claimed to improve memory (P = 0.043)

I guess these examples are quite encouraging for those who think that expensive glamour journals have had their day. Open access and open comments are the way forward.

On 2014 Dec 02, Mikkel Wallentin commented:

On the behavioral effects, the authors write: "Compared with the low-flavanol intervention, subjects who received the high-flavanol intervention showed a mean improved cognitive performance of 630 ms." However, visual inspection of the supplementary figure 3 clearly shows that the 630 ms difference at follow up is due to a modest improvement of approx. 200 ms between baseline and follow up and a much more pronounced DECLINE in the control group (approx. 400ms). Needless to say, such a difference cannot be interpreted as "a mean improved cognitive performance of 630 ms", if it can be interpreted at all.