On 2017 Jul 21, Hans Bisgaard commented:
Thank you for your interest in our study. We will be pleased to address any questions or comments in the proper scientific manner, where you submit these to the journal as a Letter to the Editor.
Sincerely
Hans Bisgaard, Bo Chawes, Jakob Stokholm and Klaus Bønnelykke COPSAC, Copenhagen, Denmark
On 2017 Jun 29, Martijn Katan commented:
My colleague Paul Brand and I published a Dutch-language comment on this paper in the Dutch Medical Journal. The English abstract is below and on www.ncbi.nlm.nih.gov/pubmed/28635579.
Abstract: Taking fish oil supplements in the third trimester of pregnancy was associated with significantly less wheezing or asthma in the child at the age of 3-5 years, according to a randomized clinical trial by Bisgaard et al., NEJM 2017. However, the results of this study should be interpreted with caution. The primary end points were modified at a late stage in the study, and two primary end points, eczema in the first 3 years of life and allergic sensitization at 18 months of age, were demoted to secondary end points, and showed no significant effect of treatment. Furthermore, the age range for the published primary end point, persistent wheeze, differed from that in the protocol. Additional concerns include the emphasis on outcomes by omega-3 fatty acid levels in the blood, a post hoc subgroup analysis not included in the protocol. In our opinion, this study does not justify advising routine fish oil supplements in pregnancy.
On date unavailable, commented:
None
On 2017 Jul 25, Konstantinos Fountoulakis commented:
Nice way to reply without replying to exact and specific questions. You already know that the NEJM editor rejected a letter by me and as i can see here he has also rejected other similar letters which raised the same questions. These specific questions seem to burn and i again mention them here:
Well i have no way of publishing this as a letter to the editor, I have already tried. To make things worse, the reply letter says (verbatim) that 'As clearly stated in the article the primary outcome was extended to distinguish wheezing children from asthmatic children'. I hope you will respond to the above issues and clarify once and for all the problem.
On 2017 Jul 21, Hans Bisgaard commented:
Thank you for your interest in our study. We will be pleased to address any questions or comments in the proper scientific manner, where you submit these to the journal as a Letter to the Editor.
Sincerely
Hans Bisgaard, Bo Chawes, Jakob Stokholm and Klaus Bønnelykke COPSAC, Copenhagen, Denmark
On 2017 Apr 20, Konstantinos Fountoulakis commented:
You do not like my critisism, and your reply is insulting. However in https://clinicaltrials.gov/ct2/show/NCT00798226 it says verbatim: 'Primary Outcome Measures: Persistent wheeze 0 to 3 years of age [ Time Frame: 3 years ]
in the paper you say (again verbatim): Primary End Point During the prespecified, double-blind follow-up period, which covered children from birth to between 3 and 5 years of age, 136 of 695 children (19.6%) received a diagnosis of persistent wheeze or asthma, and this condition was associated with reduced lung function by 5 years of age, with parental asthma, and with a genetic risk of asthma
It is quite different 0 to 3 and 0-to between 3 and 5
I would appreciate an answer on this. BTW Facebook is also good in disceminating scientific findings
On 2017 Apr 16, Hans Bisgaard commented:
Reply to comment from Konstantinos Fountoulakis:
We think that the tone and scientific level in this correspondence is inappropriate for a scientific discussion and rather resembles a facebook discussion. The primary outcome reported in the paper is identical to the registered primary outcome of asthmatic symptoms during the prespecified, double blind follow-up period until the youngest child turned 3 years of age. This primary outcome does not include any “unblinded" observation period. The definition of the primary outcome was predefined based upon a previously published algorithm using diary-registration of asthma symptoms and a predefined treatment algorithm, and also the statistical model (survival analysis by cox regression) was predefined. As evident from the paper, the analyses related to further follow-up until the youngest child turned 5 years of age, as requested by NEJM, are clearly reported separately as the results of a "continued follow-up period”.
Sincerely
Hans Bisgaard, Bo Chawes, Jakob Stokholm and Klaus Bønnelykke
COPSAC, Copenhagen, Denmark
On 2017 Apr 12, Konstantinos Fountoulakis commented:
The paper reports that high-dose supplementation with n−3 LCPUFA in the third trimester of pregnancy reduces the incidence of wheezing in the offspring [1]. However, the primary outcome as registered [2] is incidence at 3 years while in the paper it is erroneously reported as incidence between 3 and 5 years. This is highly problematic and raises a number of issues. Any changes to the protocol or to the way results are presented should had been made clear in the manuscript. Any other way of presenting the results and conclusions is problematic. It is not acceptable that the NEJM asked for an extension of the primary outcome. This could had been added as an additional post-hoc analysis. The results concerning the real primary outcome are not reported but they are probably negative, taken into consideration figure 1 and the marginal significance (p=0.03) at year 5. Furthermore, the trial becomes single blinded gradually after year 3 which makes conclusions problematic. Conclusively, the paper clearly violates the CONSORT statement [3], is probably negative concerning the primary outcome (which is in accord with the negative secondary outcomes) and it is written in a misleading way.
On 2017 Jul 21, Hans Bisgaard commented:
Thank you for your interest in our study. We will be pleased to address any questions or comments in the proper scientific manner, where you submit these to the journal as a Letter to the Editor.
Sincerely
Hans Bisgaard, Bo Chawes, Jakob Stokholm and Klaus Bønnelykke COPSAC, Copenhagen, Denmark
On 2017 Apr 06, Robert Goulden commented:
Hi Hans,
Many thanks for the reply. I really don't meant to sound incriminatig or pedantic, but the switching of primary and secondary outcomes is a widespread problem in clinical trials. Anyone looking at the history of changes to the COPSAC registration I think would be keen to find out if that had occured here.
You say 'Before unblinding of the trial we became aware that ranking of outcomes in this registration was not clear and we therefore changed this'. By that, do you mean that 'Development of eczema from 0 to 3 years of age' and 'Sensitization at 18 months of age' were mistakenly listed as primary outcomes in the original registration and subsequent revisions (until correction in Feb 2014), when your original intent was for them to be secondary outcomes from the outset? I of course understand how such an error can be made, but I hope you feel this is a reasonable question given the importance of this issue for determining the appropriate statistical significance threshold.
Rob
On 2017 Mar 26, Hans Bisgaard commented:
Reply to question from Robert Goulden
We must admit that we find this comment very incriminating and with no contribution to a scientific discussion. The primary outcome of our study was always 'wheeze’ (early asthmatic symptoms). Otherwise, we would not have reported it as such, and we doubt that the New England Journal of Medicine would have published it. Similarly, the diagnostic algorithm based upon episodes of 'troublesome lung symptoms' was pre-specified as was the analysis method (risk of developing wheeze analyzed by cox regression) in line with previous studies from our COPSAC birth cohorts. It is correct that wheeze, eczema and allergic sensitization (in that order) were all listed as ‘primary outcomes’ in the initial ClinicalTrials.gov registration. Before unblinding of the trial we became aware that ranking of outcomes in this registration was not clear and we therefore changed this (still unaware of the results of the trial). The only change after unblinding of the trial in relation to the primary outcome was the change in nomenclature to ‘Persistent wheeze or asthma’. This was due to a request from the New England Journal of Medicine of an additional 2 years follow-up from 3 to 5 years of age thereby including an age where we would normally use the term ‘asthma’.
Sincerely
Hans Bisgaard, Bo Chawes, Jakob Stokholm and Klaus Bønnelykke
COPSAC, Copenhagen, Denmark
On 2017 Feb 09, Robert Goulden commented:
Here's a letter I sent to NEJM which they declined to publish. Hopefully the authors can respond here:
A review of the history of changes on the ClinicalTrials.gov entry (NCT00798226) for Bisgaard et al.’s study raises questions about the selection of their primary outcome and the statistical significance of their positive result.
When first registered in 2008, the trial had three primary outcomes: development of wheeze, development of eczema, and sensitization. In February 2014, two months before the study completion date, the entry was edited to just have persistent wheeze as the primary outcome, with eczema and sensitisation switched to secondary outcomes. The published study in NEJM shows that persistent wheeze – presented as the sole primary outcome – was the only one of the three original primary outcomes to be statistically significant (P = 0.035).
Given multiple primary outcomes, an adjustment such as Bonferroni should have been made to the significance threshold: 0.05/3 = 0.017. Accordingly, the effect on wheeze was not statistically significant. Would the authors comment on their selection of the only ‘significant’ primary outcome as their final primary outcome? Were they aware of the study results at this point and did this influence their decision?
On 2017 Feb 09, Robert Goulden commented:
Here's a letter I sent to NEJM which they declined to publish. Hopefully the authors can respond here:
A review of the history of changes on the ClinicalTrials.gov entry (NCT00798226) for Bisgaard et al.’s study raises questions about the selection of their primary outcome and the statistical significance of their positive result.
When first registered in 2008, the trial had three primary outcomes: development of wheeze, development of eczema, and sensitization. In February 2014, two months before the study completion date, the entry was edited to just have persistent wheeze as the primary outcome, with eczema and sensitisation switched to secondary outcomes. The published study in NEJM shows that persistent wheeze – presented as the sole primary outcome – was the only one of the three original primary outcomes to be statistically significant (P = 0.035).
Given multiple primary outcomes, an adjustment such as Bonferroni should have been made to the significance threshold: 0.05/3 = 0.017. Accordingly, the effect on wheeze was not statistically significant. Would the authors comment on their selection of the only ‘significant’ primary outcome as their final primary outcome? Were they aware of the study results at this point and did this influence their decision?
On 2017 Mar 26, Hans Bisgaard commented:
Reply to question from Robert Goulden
We must admit that we find this comment very incriminating and with no contribution to a scientific discussion. The primary outcome of our study was always 'wheeze’ (early asthmatic symptoms). Otherwise, we would not have reported it as such, and we doubt that the New England Journal of Medicine would have published it. Similarly, the diagnostic algorithm based upon episodes of 'troublesome lung symptoms' was pre-specified as was the analysis method (risk of developing wheeze analyzed by cox regression) in line with previous studies from our COPSAC birth cohorts. It is correct that wheeze, eczema and allergic sensitization (in that order) were all listed as ‘primary outcomes’ in the initial ClinicalTrials.gov registration. Before unblinding of the trial we became aware that ranking of outcomes in this registration was not clear and we therefore changed this (still unaware of the results of the trial). The only change after unblinding of the trial in relation to the primary outcome was the change in nomenclature to ‘Persistent wheeze or asthma’. This was due to a request from the New England Journal of Medicine of an additional 2 years follow-up from 3 to 5 years of age thereby including an age where we would normally use the term ‘asthma’.
Sincerely
Hans Bisgaard, Bo Chawes, Jakob Stokholm and Klaus Bønnelykke
COPSAC, Copenhagen, Denmark
On 2017 Apr 12, Konstantinos Fountoulakis commented:
The paper reports that high-dose supplementation with n−3 LCPUFA in the third trimester of pregnancy reduces the incidence of wheezing in the offspring [1]. However, the primary outcome as registered [2] is incidence at 3 years while in the paper it is erroneously reported as incidence between 3 and 5 years. This is highly problematic and raises a number of issues. Any changes to the protocol or to the way results are presented should had been made clear in the manuscript. Any other way of presenting the results and conclusions is problematic. It is not acceptable that the NEJM asked for an extension of the primary outcome. This could had been added as an additional post-hoc analysis. The results concerning the real primary outcome are not reported but they are probably negative, taken into consideration figure 1 and the marginal significance (p=0.03) at year 5. Furthermore, the trial becomes single blinded gradually after year 3 which makes conclusions problematic. Conclusively, the paper clearly violates the CONSORT statement [3], is probably negative concerning the primary outcome (which is in accord with the negative secondary outcomes) and it is written in a misleading way.
On 2017 Jun 29, Martijn Katan commented:
My colleague Paul Brand and I published a Dutch-language comment on this paper in the Dutch Medical Journal. The English abstract is below and on www.ncbi.nlm.nih.gov/pubmed/28635579.
Abstract: Taking fish oil supplements in the third trimester of pregnancy was associated with significantly less wheezing or asthma in the child at the age of 3-5 years, according to a randomized clinical trial by Bisgaard et al., NEJM 2017. However, the results of this study should be interpreted with caution. The primary end points were modified at a late stage in the study, and two primary end points, eczema in the first 3 years of life and allergic sensitization at 18 months of age, were demoted to secondary end points, and showed no significant effect of treatment. Furthermore, the age range for the published primary end point, persistent wheeze, differed from that in the protocol. Additional concerns include the emphasis on outcomes by omega-3 fatty acid levels in the blood, a post hoc subgroup analysis not included in the protocol. In our opinion, this study does not justify advising routine fish oil supplements in pregnancy.