On 2017 May 01, Kevin Hall commented:

The corrected manuscript is now posted online, including the Supplemental Materials describing the methodology for the systematic review and meta-analysis.

On 2017 Mar 22, Kevin Hall commented:

Dr. Harnke is correct that the early online publication did not provide the peer-reviewed Supplemental Materials describing the methodology for the systematic review and meta-analysis. Also, the online publication erroneously provided the penultimate version of the figures. The Supplemental Materials and the updated figures are available upon request: kevinh@niddk.nih.gov.

On 2017 Mar 20, Ben Harnke commented:

The E-pub ahead of print version of this article does not appear to provide details about the search strategy, databases searched, limits, etc. used to identify the included studies. Without this information it is impossible to replicate the study or to verify that all relevant citations were located. Hopefully these details will be included in the final published version and/or online supplementary material.

On 2017 Nov 08, Cicely Saunders Institute Journal Club commented:

We selected and discussed this paper at our monthly journal club on 1st November 2017.

The paper generated a lot of discussion and we felt that this was an important concept, especially for clinicians, to think about. The topic of QALYs was unfamiliar to some of us and we found that the authors explained it very clearly in the paper. We were intrigued by the use of an integrative review method and discussed this at length. It may have been helpful to read more explanation of this method and know how it differs from other types of review methods. We also wondered about some of the inclusion/exclusion criteria such as the exclusion of reviews and the decision making process for the theoretical papers included. We enjoyed discussing the themes which emerged from this paper and the wider debate around the most appropriate measures for palliative care populations, particularly in light of the recent paper by Dzingina et al. 2017 (https://www.ncbi.nlm.nih.gov/pubmed/28434392). We feel this paper will be a useful educational resource.

Commentary by Dr. Nilay Hepgul & Dr. Deokhee Yi on behalf of researchers at Cicely Saunders Institute of Palliative Care, Policy & Rehabilitation, King’s College London.

On 2017 May 04, Ralf Koebnik commented:

This publication states that “HprX is the AraC/XylS regulator of the Xanthomonas citri T3SS and is activated by HrpG, a sensor kinase that phosphorylates HrpX [43].“ This is a wrong interpretation of Ref. 43. To be correct, it is HrpX, not HprX, and second, HrpX is not activated via phosphorylation by HrpG. Instead, HrpG (probably in a phorphorylated state, but this is still speculation) activates the transcription of the hrpX gene. HrpX in turn binds to the promoter region of hrp genes that encode the structural components of the T3SS.

On 2017 Mar 12, John Tucker commented:

The article summarizes the results of Ohio’s 2011-2015 program to reduce prescription painkiller overdose deaths by stating that prescribing was reduced by 10%, leading to a reduction in the percentage of drug overdose deaths attributable to prescription painkillers from 45% to 22%.

While it would be natural for a reader to assume that this percentage reduction arose from a decline in prescription drug overdoses, this is not the case. Instead, overdoses due to prescription painkillers remained relatively constant while heroin and illicit fentanyl deaths skyrocketed.

CDC WONDER gives the following death counts for Ohio in 2011 and 2015

Heroin: 325 in 2011 and 1103 in 2015

Other Opioids: 197 in 2011 and 340 in 2015

Methadone: 70 in 2011 and 50 in 2015

Other synthetic narcotics (including fentanyl): 57 in 2011 and 891 in 2015

Unspecified narcotics: 62 in 2011 and 80 in 2015

Total opioid overdose deaths: 711 in 2011 and 2464 in 2015.

Thus the opioid overdose death rate in Ohio increased by 246% during the analysis period, compared to 40% for the United States as a whole.

The program cannot by any stretch of the imagination be considered a success, and simply serves as yet another example of the futility of supply-side focused approaches to drug abuse.

On 2017 Mar 04, Andrea Messori commented:

PubMed database: A selection of pharmacoeconomic studies based on the net monetary benefit

Andrea Messori, HTA Unit, Regional Health Service, Firenze, Italy

The objective of the study by Capri and co-workers was to compare the cost-effectiveness of pazopanib versus sunitinib as first-line therapy in patients with advanced or metastatic renal cell carcinoma; the perspective was that of the Italian National Health Service.

In patients with cancer, most economic studies based on this design are carried out by determining the incremental cost effectiveness ratio (ICER). In contrast, one reason of interest of the study by Capri and co-workers is that the net monetary benefit (NMB) was the methodological tool employed to carry out the pharmacoeconomic analysis.

Although the NMB is not the standard tool for performing these analyses, there are some advantages in using this parameter as opposed to the ICER. For example, while the relationship between the cost of the intervention and the ICER is nonlinear, the relationship between the cost of the intervention and the NMB is linear. Hence, predicting the consequences of an increased cost of treatment (or a decreased cost of treatment) is easier, or more intuitive, if the NMB is used rather than the ICER.

Using a standard syntax of PubMed search (“net monetary benefit”[text]; search date = 4 March 2017), we identified a total of 148 citations that met this criterion. Among these citations, we selected 20 studies published between 2010 and 2016 in which the NMB played a key role in generating the pharmacoeconomic results (see http://www.osservatorioinnovazione.net/papers/nmb20examples.html).

Curiously enough, among these 148 citations, some studies were lacking in which the NMB had been successfully employed (e.g. Ganesalingam J, Pizzo E, Morris S, Sunderland T, Ames D, Lobotesis K.Cost-Utility Analysis of Mechanical Thrombectomy Using Stent Retrievers in Acute Ischemic Stroke. Stroke. 2015 Sep;46(9):2591-8, https://www.ncbi.nlm.nih.gov/pubmed/26251241/); this indicates that the keyword “net monetary benefit” in PubMed misses a number of pertinent articles.

In conclusion, despite the low number of retrieved articles, our preliminary overview of the literature shows that the NMB is still being used in pharmacoeconomic studies and deserves to be further investigated.

On 2017 Apr 05, Julie Glanville commented:

As we note in the Methods section of our paper, this paper is informed by an update of a systematic review:

"For the update to the systematic review, we searched 18 databases, websites, trial registries and three conference websites between February and March 2016. The search strategy for MEDLINE is shown in Supplemental Fig. 1 (Fig. S1) and the other searches are available on request. This search strategy was originally developed in 2013, and then updated for this analysis taking into account relevant recent changes in indexing [19, 20]. The searches were not limited by date, language, or document type. The information sources searched are shown in Supplemental Table 1 (Table S1). Search results were downloaded into Endnote and de-duplicated against each other and against the results of the original review."

Figure 1 in the supplementary material shows the Medline update search. Although it is a search carried out to identify new records since the original searches, we did not limit to recent years but reran the search for all years. This resulted in 6845 records. However, many of these records had been processed in the initial SR, so "Search results were downloaded into Endnote and de-duplicated against each other and against the results of the original review". This resulted in a much lower number of records which needed to be assessed for relevance from Medline in the update as can be seen in Table S2. In Table S2 the second column shows the number of records downloaded before deduplication against the original search results and the third column shows the number of records assessed for relevance after deduplication against the original search results. Hence the number difference. We acknowledge that there has been a transposition error for the Medline results in Table S2 - the search resulted in 6845 records and we have entered 2845 by mistake. We will correct the transposition error.

Despite what we say in the Methods section, the full search strategies for the original search and the update searches are in the supplementary material at page 28 onwards.

On 2017 Apr 04, Wichor Bramer commented:

The authors of this article seen to have made a mistake in their registration of results per database. The supplements show that EMBASE alone retrieved over 7000 results and SCI more than 6000. Yet the total number of articles after deduplication is 5500.

Only the Medline search is provided in detail. The number of results shown in the search strategy seems to be 6800, whereas in the overview table of all databases the number is 2800.

It is recommended to add the search strategies for all databases and to keep a good and clear track of the results per database before and after deduplication.

On 2017 Jul 25, Donald Forsdyke commented:

LECTIN PATHWAY STUDIES WITH PLANT MANNOSE-BINDING LECTINS

Papers on the lectin pathway (LP) of complement activation in animal sera generally refer to animal mannose-binding lectins (MBLs), with little reference to work with plant MBLs. For example, citing May and Frank (1973), this fine paper states: "Reports of unconventional complement activation in the absence of C4 and/or C2 predate the discovery of LP." Actually, a case can be made that the discovery of the LP predates May-Frank.

The MASP-binding motif on animal MBL, which is necessary for complement activation, includes the amino acid sequence GKXG (at positions 54-57), where X is often valine. The plant lectin concanavalin-A (Con-A) has this motif at approximately the same position in its sequence (the 237 amino acid subunit of Con-A had the sequence GKVG at positions 45-48). The probability of this being a chance event is very low. Indeed, prior to the discovery of MASP involvement, Milthorp & Forsdyke (1970) reported the dosage-dependent activation of complement by Con-A.

As far as I am aware, it has not been formally shown that MASP is involved in the activation of the complement pathway by this plant MBL. Our studies in the 1970s demonstrated that Con-A activates complement through a cluster-based mechanism, which is consistent with molecular studies of animal MBL showing “juxtaposition- and concentration dependent activation” (Degn et al. 2014). References to our several papers on the topic may be found in a review of innate immunity (Forsdyke 2016).

Degn SE et al. (2014) Complement activation by ligand-driven juxtaposition of discrete pattern recognition complexes. Proc Natl Acad Sci USA 111:13445-13450. Degn SE, 2014

Forsdyke DR (2016) Almroth Wright, opsonins, innate immunity and the lectin pathway of complement activation: a historical perspective. Microb Infect 18: 450-459. Forsdyke DR, 2016

May JE, Frank MM (1973) Hemolysis of sheep erythrocytes in guinea pig serum deficient in the fourth component of complement. I. antibody and serum requirements. J Immunol 111: 1671-1677. May JE, 1973

Milthorp PM, Forsdyke DR (1970) Inhibition of lymphocyte activation at high ratios of concanavalin A to serum depends on complement. Nature 227:1351-1352 Milthorp P, 1970

Yaseem et al. (2017) Lectin pathway effector enzyme mannan-binding lectin-associated serine protease-2 can activate native complement C3 in absence of C4 and/or C2. FASEBJ 31:2210-2219 Yaseen S, 2017

On 2017 Feb 18, Clive Bates commented:

So we learn from this study that pharmacists demand:

training in the form of information packs (88%), online tutorials (67%), continuous professional development (CPD) workshops (43%) to cover safety, counselling, dosage instructions, adverse effects and role in the smoking cessation care pathway in the future.

But how many of them have made use of the existing resources already provided by the UK National Centre for Smoking Cessation and Training, in particular, its excellent E-cigarettes: a briefing for stop smoking services, 2016. This is readable and accessible and easily found by anyone with a professional interest.

If they wanted to go into the issue more deeply, there is the Royal College of Physicians report, Nicotine without smoke: tobacco harm reduction, 2016 which provides a scientific assessment for UK health professionals, and concludes:

that e-cigarettes are likely to be beneficial to UK public health. Smokers can therefore be reassured and encouraged to use them, and the public can be reassured that e-cigarettes are much safer than smoking.

As they are selling these products, isn't there a legitimate professional expectation that community pharmacists should make a modest effort to find out more about them? The survey reveals a disturbing level of ignorance and unscientific assertion and the demand for more training is the flip side of an admission of ignorance. A good question would have found out whether they have made any effort at all to resolve their uncertainties, for example by consulting the sources above.

On 2017 Feb 12, Romain Brette commented:

From the perspective of a computational neuroscientist, I believe a very important point is made here. Models are judged on their ability to account for experimental data, so the critical question is what counts as relevant data? Data currently used to constrain models in systems neuroscience are most often neural responses to stereotypical stimuli, and results from behavioral experiments with well-controlled but unecological tasks, for example conditioned responses to variations in one dimension of a stimulus.

In sound localization for example, one of the four examples in this essay, a relevant problem for a predator or a prey is to locate the source of a sound, i.e. absolute localization. But models such as the recent model mentioned in the essay (which is influential but not consensual) have been proposed on the basis of their performance in discriminating between identical sounds played at slightly different angles, a common experimental paradigm. Focusing on this paradigm leads to models that maximize sensitivity, but perform very poorly on the more ecologically relevant task of absolute localization, which casts doubts on the models (Brette R, 2010; Goodman DF, 2013). Unfortunately the available set of relevant behavioral data is incomplete (e.g., what is the precision of sound localization with physical sound sources in ecological environments, and are orienting responses invariant to non-spatial aspects of sounds?). Thus I sympathize with the statement in this essay that more proper behavioral work should be done.

In other words, a good model should not only explain laboratory data: it should also work (i.e. explain how the animal manages to do what it does). It is good to remind this crucial epistemological point.

On 2017 Apr 03, Dmitri Rusakov commented:

We thank the Reviewer for following up the story. Below are our point-by-point reply to his latest set of comments. This reply and the manuscript version revised accordingly appeared to satisfy the journal Editorial Board and the other reviewer(s). We appreciate that this reviewer might not have therefore received the full set of explanations shown below. We however urge him to look at the published paper and its Supplementary data as these do contain material answers to his key questions.

Reviewer #1

Q: The authors have taken a rather minimalist approach to my suggestion that the fluorescence anistropy measurements be analysed and presented in greater detail in the MS.

A: There must have been a misunderstanding. In response to the original Reviewer's comments, we have provided a full point-by-point response with extended explanations, added quantitative evidence for the two-exponential approximation, and provided a full summary Table for the FLIM characteristics over six different areas of interest. This is precisely what was requested in the original comments.

Q: In addition, in considering this revision, additional questions have arisen. I therefore give a more detailed and prescriptive list of the data that needs to be shown. The following conditions are of interest: 1) free solution 2) with extracellular dye a) measurement inside cell b) measurement over neuropil c) measurement over synapse d) measurement inside pipette 3) with intracellular dye a) measurement inside cell 4) no dye a) measurement of autofluorescence over neuropil b) measurement inside soma For each of the above conditions, please show: 1) full fluorescence time course -1 to 12 ns 2) full anisotropy time course -1 to 12 ns 3) specimen traces 4) global averages 5) fit of global average 6) timing of the light pulse should always be indicated (I assume it occurs at 1ns, but this must be made explicit)

A: We note that all the requested information is contained, in the shape of single-parameter outcomes, in the original figures and Tables. We also note that in healthy brain slices autofluorescence (two-photon excitation) is undetectable. With all due respect, we did not fully understand the grounds for requesting excessive primary material: the process of analysing anisotropy FLIM data involves automated, pixel-by-pixel data collection and curve fittings representing tens of thousands of single-pixel plots at all stages of the data processing. Presenting such data does not appear technically feasible. However, we have added some extensive primary-data examples, as requested, to illustrate:

(a) Fluorescence decay in parallel and perpendicular detectors at different viscosity values (Fig. S1a);

(b) Instrument response for the two-detector system (Fig. S1b), indicating that it has much faster dynamics than the anisotropy decay;

(c) Anisotropy decay data in slice tissue after dye washout - indicating a specific reduction of the fast (free-diffusion) rather than slow (membrane-bound) molecular component (Fig. S1c);

(d) AF350 anisotropy decay examples recorded in a free medium, intracellular compartment, extracellular space in the synapse and neuropil extracellular space (Fig. S1d).

Q: It remains a good idea to try the same measurements with a second dye.

A: AF350 is the smallest bright fluorophore which shows no lifetime dependency on physiological cellular environment. It is therefore the best candidate to explore the movement of small ions or neurotransmitter molecules such as glutamate in similar environment. We have tried other fluorophores such as AF594, which is three times heavier, more prone to photobleaching and more likely to bind to cell membranes in slices, all of which makes interpreting the data more difficult. We believe that the experimental evidence obtained in the present study has led to a self-contained set of conclusions that are of interest to the journal readership. Repeating the entire study with a different dye, or a different animal species, or a different preparation, or else, might be a good idea for future research.

Q: Why is the rise of the anisotropy not instantaneous in Fig. 1B? I couldn't find any mention of temporal filtering.

A: The rising phase is influenced by the instrument response to the femtosecond laser pulse in raw lifetime data, which remained unmodified in the presented plots. Signal deconvolution would produce instantaneous anisotropy (while increasing noise in raw data) which is not our preferred way of presenting the data. We have added this explanation to the text.

Q: Assuming that the light pulse occurs at 1ns in Fig. S1A, why is the anisotropy shown so high? I would have thought that it should have decayed over a time constant (to about 1/e) by the point where the data shown, yet it is only reduced by about 15%. Was some additional normalisation carried out that I missed? If so it should certainly be removed and the true time course shown.

A: The example in question shows a direct comparison between decay shapes in baseline conditions and after photobleaching: for illustration purposes, the graph displays a fragment of raw decay data, including the instrument response (without deconvolution). The latter at least doubles the apparent decay constant, plus the fast component has a y-offset of 0.2-0.3 due to the slow-component. These concomitants make the fast decay appear slower but this is irrelevant for the purposes of this particular raw data illustration (in contrast, Table S1 summary data are obtained with the instrument response de-convolved and removed). The text and figure legend has been expanded to explain this.

Q: In case it is not clear, I am interested in the possibility of the fast component in fact containing two components. The data do not allow me to evaluate this possibility.

A: Whilst we appreciate personal scientific interests of the Reviewer, we see no scientific reasons, in the present context, to try and find 'the possibility' of fast anisotropy decay sub-components: we simply refer to all molecular sub-populations showing distinctly fast anisotropy decay as one free-diffusion pool.

Q: Does Scientific Reports required the authors to provide access to the original data upon publication? What is the authors' position on this?

A: All original data, including tens of thousands of single-pixel FLIM data plots at different analysis stages, etc. are available from the authors on request.

On 2017 Mar 25, Boris Barbour commented:

Below I show the key parts of my first and second reviews of this paper, which reports a very interesting and powerful optical technique for probing the microscopic properties of fluid compartments in the brain. I felt that greater detail of the analysis should be shown to support fully the conclusion of slowed diffusion in the extracellular space and synaptic cleft. It will be apparent that some questions unfortunately only occurred to me upon reading the first revision. The authors only responded to some of the points raised before the paper was published without my seeing it again. In particular, no global average anisotropy time courses are shown and the timing of the excitation pulses remains rather mysterious.

First review

This MS reports an extremely interesting approach to providing quantitative information about the diffusion of small molecules in micro-compartments of brain tissue - potentially resolving the intracellular and extracellular spaces, as well as providing information about diffusion within the synaptic cleft.

The basis for the approach is to measure the relaxation of fluorescence polarisation following two-photon excitation of small compartments. If polarised exciting light is used the emitted light is also polarised, as long as the orientation of the fluorophore remains unchanged. However, as the molecule undergoes thermal reorientations, that polarisation is lost. The authors use this technique to measure the diffusion of a small molecule - alexa fluor 350.

A subsequent section of the MS reports some synaptic modelling, applying the tissue/free ratio obtained for the fluorophore to the modelled glutamate. However, the important and by far the most interesting part of the MS is the diffusion measurement. I would be happy for the MS to consist solely of an expanded and more detailed analysis of these measurements, postponing the modelling to another paper.

My main comments relate to the analysis, presentation and interpretation of these diffusion measurements. The authors report that the relaxation time for the polarisation displays two phases - a rapid phase, which is somewhat slowed in brain tissue, and a slow phase attributed to membrane binding. But the authors do not illustrate the analysis of the fast component. As this is critical, a good deal more detail should be shown.

A first issue is whether there are additional bound/retarded states other than "fixed". To examine this the authors should show the fit to the average relaxation; it is important to be able to verify that there is a single exponential fast component rather than some mixture (it may not be possible to tell with any certainty). Some examination of the robustness and precision of the fitting would also be desirable.

The authors should also characterise the variation between different measurements of the same compartments. The underlying question here is how the various decay components might vary as, for instance, the ratio of membrane to extracellular space varies across different measurement points.

I think the authors need to give more thought to the possibility that some of the slowing they observe arises from fluorophore embedded in the membrane without being immobilised. I don't see how this can be easily ruled out - certainly the two-photon resolution does not permit distinguishing the membrane and fluid phases in the neuropil. Additionally, how would the authors rule out adsorption onto some extracellular proteins?

The reason I raise these points is because I have at least a slight difficulty with the interpretation. A slowing of molecular rotation of 70% (intracellular) suggests to me that a large fraction of fluorophores, essentially 100%, must be in direct contact with some larger molecule. This seems quite extreme even in the crowded intracellular space. I have similar reservations about the synaptic cleft and extracellular space. Is at least 50% and 30% of the volume of these spaces really occupied by macromolecules? There may be somewhat reduced diffusion due to a boundary layer at the membrane or near macromolecules. Do estimates of the thickness of such a boundary layer exist (and its effect on diffusion)?

Second review

The following conditions are of interest:

1) free solution

2) with extracellular dye

a) measurement inside cell

b) measurement over neuropil

c) measurement over synapse

d) measurement inside pipette

3) with intracellular dye

a) measurement inside cell

4) no dye

a) measurement of autofluorescence over neuropil

b) measurement inside soma

For each of the above conditions, please show:

1) full fluorescence time course -1 to 12 ns

2) full anisotropy time course -1 to 12 ns

3) specimen traces

4) global averages

5) fit of global average

6) timing of the light pulse should always be indicated (I assume it occurs at 1ns, but this must be made explicit)

It remains a good idea to try the same measurements with a second dye.

Why is the rise of the anisotropy not instantaneous in Fig. 1B? I couldn't find any mention of temporal filtering.

Assuming that the light pulse occurs at 1ns in Fig. [1E], why is the anisotropy shown so high? I would have thought that it should have decayed over a time constant (to about 1/e) by the point where the data shown, yet it is only reduced by about 15%. Was some additional normalisation carried out that I missed? If so it should certainly be removed and the true time course shown.

In case it is not clear, I am interested in the possibility of the fast component in fact containing two components. The data do not allow me to evaluate this possibility.

(Note to moderators: as the copyright holder of my reviews, I am entitled to post them.)

On 2017 May 10, Fernanda Leite commented:

https://www.nature.com/nature/journal/v542/n7640/full/nature21363.html#comment-69823

On 2017 Jul 24, Jakob Näslund commented:

For a discussion of this study regarding some outstanding issues relating to methodology, as well as the presence of a number of possible inaccuracies, see our commentary in Acta Neuropsychiatrica entitled "Multiple possible inaccuracies cast doubt on a recent report suggesting selective serotonin reuptake inhibitors to be toxic and ineffective", available at https://doi.org/10.1017/neu.2017.23

On 2017 Jul 24, Konstantinos Fountoulakis commented:

This paper confirms that the overall SMD is around or above 0.30 adding to the literature against the idea that antidepressants do not work (e.g. Kirsch 2008). The question of the magnitude of the effect has been discussed in the literature again and again. The NICE has abandoned since years the 3-point magnitude to define 'clinical relevance' and a SMD of 0.30 is the effect size expected for successful psychiatric treatments and also for treatments elsewhere in medicine. Of course we would like more, but this is the only means we have. All the other options (psychotherapy, alternative therapies etc) do not meet the stringent criteria of this meta-analysis as there are essentially not blinded and not adequately placebo-controlled, not to mention the risk of bias. I strongly disagree with the comment on the HDRS by the authors. Yes, regulatory authorities do recommend the HDRS but this does not constitute an essential argument. It is not correct to double register an event as both a symptom and an adverse event. It is either or (at least in principle). Unfortunately the HDRS is based on an antiquitated model of depression while the MADRS is tailored to the needs of trials. For a review please see Fountoulakis et al J Psychopharmacol. 2014 Feb;28(2):106-17. Concerning the adverse events, yes indeed there is a significant effect of the active drug but the NNH are high and there was no difference in severe adverse events in comparison with placebo. In my opinion, the real SMD of SSRIs are much higher, but not really impressive. This is masked by the properties of the HDRS but also by the possibility that a large number of patients enrolled in these studies are not suitable for a number of reasons. I would love to see US vs, rest of world comparison

On 2017 Jun 08, Christian Gluud commented:

Response to Søren Dinesen Østergaard's critique

Søren Dinesen Østergaard (SDØ) [1] criticizes our systematic review on selective serotonin reuptake inhibitors (SSRI) for patients with major depressive disorder [2] for using Hamilton’s depression rating scale (HDRS)17 instead of HDRS6. SDØ refer to four studies ‘documenting’ his claims [3-6].

Two of the studies relate to duloxetine and desvenlafaxine, which are dual action drugs and not SSRIs [3, 4]. The third study is a meta-analysis assessing fluoxetine versus placebo [5]. The results show a mean effect size of the SSRI of -0.30 (95% confidence interval (CI) -0.39 to -0.21) when using HDRS17 and an effect size of -0.37 (95% CI -0.46 to -0.28) when using HDRS6. The difference of 0.07 corresponds to 0.7 HDRS17 points assuming a standard deviation of 10 points. The fourth study is a patient-level analysis of 18 industry-sponsored placebo-controlled trials regarding paroxetine, citalopram, sertraline, or fluoxetine [6]. The authors report a mean effect size of the SSRIs of -0.27 when using HDRS17 and an effect size of -0.35 when using HDRS6 [6]. The difference of 0.08 corresponds to 0.8 HDRS17 points assuming a standard deviation of 10 points. Hence, the absolute effect size difference between the two scales seems less than 1 HDRS point. The National Institute for Clinical Excellence (NICE) recommended a difference of 3 points on the HDRS17 for 'a minimal effect' [7-9]. However, the required minimal clinical relevant difference is probably much larger than this figure. One study showed that a mirtazapine-placebo mean difference of up to 3.0 points on the HDRS corresponds to ‘no clinical change’ [10]. Another study showed that a SSRI-placebo mean difference of 3.0 points is undetectable by clinicians, and that a mean difference of 7.0 HDRS17 points is required to correspond to a rating of ‘minimal improvement’ [11].

Moreover, none of the meta-analyses [5, 6] take into account risks of systematic errors (‘bias’) [12-14] or risks of random errors [15]. Hence, there are risks that the two meta-analyses may overestimate the beneficial effects of SSRIs.

Other studies have shown that HDRS17 and HDRS6 largely show similar results [16,17]. It cannot be concluded that HDRS6 is a better assessment scale than HDRS17, just considering the psychometric validity of the two scales. If the total score of HDRS17 is affected by some of the adverse effects of SSRIs, then this might in fact better reflect the actual summed clinical effects of SSRIs than HDRS6 ignoring these effects. Until scales are validated against patient-centred clinically relevant outcomes, such scales are merely non-validated surrogate outcomes [18].

National and international medical agencies [19-21] all recommend HDRS17 for assessing depressive symptoms. We need access to all individual patient data from all randomised clinical trials to compare the effects of antidepressants on HDRS6 to HDRS17 [22].

SDØ states “no conflicts of interest“. We are aware that SDØ has received substantial support from ‘Lundbeckfonden’, its main objective being to maintain and expand the activities of the Lundbeck Group, one of the companies producing and selling SSRIs [23]. We think it would have been fair to declare this.

Janus Christian Jakobsen, Kiran Kumar Katakam, Naqash Javaid Sethi, Jane Lindshou, Jesper Krogh, and Christian Gluud.

Conflicts of interest:
 None known.

Copenhagen Trial Unit, Centre for Clinical Intervention Research, Rigshospitalet, Copenhagen, Denmark

References 1. Ostergaard SD: Do not blame the SSRIs: blame the Hamilton Depression Rating Scale. Acta Neuropsychiatrica 2017:1-3. 2. Jakobsen JC, Katakam KK, Schou A, et al: Selective serotonin reuptake inhibitors versus placebo in patients with major depressive disorder. A systematic review with meta-analysis and Trial Sequential Analysis. BMC Psychiatr 2017, 17(1):58. 3. Bech P, Kajdasz DK, Porsdal V: Dose-response relationship of duloxetine in placebo-controlled clinical trials in patients with major depressive disorder. Psychopharmacology 2006, 188(3):273-280. 4. Bech P, Boyer P, Germain JM, et al: HAM-D17 and HAM-D6 sensitivity to change in relation to desvenlafaxine dose and baseline depression severity in major depressive disorder. Pharmacopsychiatry 2010, 43(7):271-276. 5. Bech P, Cialdella P, Haugh MC, et al: Meta-analysis of randomised controlled trials of fluoxetine v. placebo and tricyclic antidepressants in the short-term treatment of major depression. Br J Psychiatr 2000, 176:421-428. 6. Hieronymus F, Emilsson JF, Nilsson S, Eriksson E: Consistent superiority of selective serotonin reuptake inhibitors over placebo in reducing depressed mood in patients with major depression. Mol Psychiatr 2016, 21(4):523-530. 7. Fournier JC, DeRubeis RJ, Hollon SD, et al: Antidepressant drug effects and depression severity: a patient-level meta-analysis. JAMA 2010, 303(1):47-53. 8. Mathews M, Gommoll C, Nunez R, Khan A: Efficacy and safety of vilazodone 20 and 40 Mg in major depressive disorder: a randomized, double-blind, placebo-controlled trial. Int Clin Psychopharmacol 2015, 30. 9. Kirsch I, Deacon BJ, Huedo-Medina TB, et al: Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS medicine 2008, 5(2):e45. 10. Leucht S, Fennema H, Engel R, et al: What does the HAMD mean? J Affect Disord 2013, 148(2-3):243-248. 11. Moncrieff J, Kirsch I: Empirically derived criteria cast doubt on the clinical significance of antidepressant-placebo differences. Cont Clin Trials 2015, 43:60-62. 12. Hróbjartsson A, Thomsen ASS, Emanuelsson F, et al: Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ : Canadian Medical Association Journal = Journal de l'Association Medicale Canadienne 2013, 185(4):E201-211. 13. Lundh A, Lexchin J, Mintzes B, Scholl JB, Bero L: Industry sponsorship and research outcome. Cochrane Database Syst Rev 2017, Art. No.: MR000033. DOI: 10.1002/14651858.MR000033.pub3.(2):MR000033. 14. Savovic J, Jones HE, Altman DG, et al: Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 2012, 157(6):429-438. 15. Wetterslev J, Jakobsen JC, Gluud C: Trial Sequential Analysis in systematic reviews with meta-analysis. BMC Medical Research Methodology 2017, 17(1):39. 16. Hooper CL, Bakish D: An examination of the sensitivity of the six-item Hamilton Rating Scale for Depression in a sample of patients suffering from major depressive disorder. J Psychiatry Neurosci 2000, 25(2):178-184. 17. O'Sullivan RL, Fava M, Agustin C, Baer L, Rosenbaum JF: Sensitivity of the six-item Hamilton Depression Rating Scale. Acta psychiatrica Scandinavica 1997, 95(5):379-384. 18. Gluud C, Brok J, Gong Y, Koretz RL: Hepatology may have problems with putative surrogate outcome measures. J Hepatol 2007, 46(4):734-742. 19. Sundhedsstyrelsen (Danish Health Agency): Referenceprogram for unipolar depression hos voksne (Guideline for unipolar depression in adults). http://wwwsstdk/~/media/6F9CE14B6FF245AABCD222575787FEB7ashx 2007. 20. European Medicines Agency: Guideline on clinical investigation of medicinal products in the treatment of depression. EMA/CHMP/185423/2010 Rev 2 previously (CPMP/EWP/518/97, Rev 1) 2013. 21. U.S. Food and Drug Administration: https://www.fda.gov/ohrms/dockets/AC/07/briefing/2007-4273b1_04-DescriptionofMADRSHAMDDepressionR(1).pdf. 22. Skoog M, Saarimäki JM, Gluud C, et al.: Transaprency and Registration in Clinical Research in the Nordic Countries. Nordic Trial Alliance, NordForsk. 2015:1-108. 23. Lundbeck Foundation. http://www.lundbeckfonden.com/about-the-foundation.25.aspx.

On 2017 Mar 03, Søren Dinesen Østergaard commented:

For a comment on this meta-analysis, see the letter to the editor in Acta Neuropsychiatrica "Do not blame the SSRIs: blame the Hamilton Depression Rating Scale": https://doi.org/10.1017/neu.2017.6

SDØ declares no conflicts of interest. Grants received from the Lundbeck foundation (see comment below) were all non-conditional and not given to support studies on antidepressant efficacy.

On 2017 Aug 06, Andrea Messori commented:

Clinical and administrative issues in the in-hospital management of innovative pharmaceuticals and medical devices

by Andrea Messori and Valeria Fadda

ESTAR, Regional Health System, via San Salvi 12, 50100 Firenze (Italy)​

Innovative treatments are increasingly being developed for a variety of disease conditions, particularly in the field of pharmaceutics and medical devices. In this scenario, two needs have clearly emerged in the past years: firstly, the activities of horizon scanning, assessment of innovation, and prediction of health-care expenditure for the new products are becoming more and more important in practical terms, thus underscoring that all the main components HTA must be strengthened to improve the governance of in-hospital innovation; secondly, in most jurisdictions of national health systems (NHS), the management of innovation includes also the administrative process of procurement, and so this mix of clinical and bureaucratic pathways further complicates the practical handling of the new products. On the other hand, optimizing the supply chain and the procurement of pharmaceuticals and/or medical devices is known to be essential for the governance of public health-care [1].

To our knowledge, the current medical literature does not comprise any real-life experience in which the management of new products for in-hospital use has been described in the context of a public NHS by examining both clinical and administrative aspects. In this brief report, we present one such experience that has been carried out in 2017 in a regional setting (the Tuscany region) of the Italian NHS.

The Tuscany region includes a total of 3.75 million inhabitants; there are 7 separate Local Health Authorities with an overall number of 27 hospitals and 11,000 beds.

Since 2014, the requests for any new pharmaceutical or medical device by the regional hospitals are submitted through a regional website. The activity carried out in May 2017 (601 requests) has been taken as an example.

If one examines separately the data for pharmaceuticals (N=436) and medical devices (N=165), the reasons for these requests were of administrative nature in 42% of cases for pharmaceuticals and in 84% of cases for devices; in more detail, administrative requests dealt with the need to extend a contract close to expiration or to run a new tender including some new products in replacement for the old tender. New products were requested in 38% and in 16% of cases for pharmaceuticals and devices, respectively, but most of these new products could not be classified as innovative (according to the criteria of our national medicines agency [2]). Overall, there were only 3 innovative products -two medical devices and one drug-, and their requests, according to our internal procedures, were transmitted to the Regional Unit responsible for HTA reports. These 3 products that met our criteria for innovation represented only 0.5% of all requests received at our website.

In conclusion, in the perspective of public hospitals, our experience shows that the management of innovation raises several practical problems because clinical and administrative aspects often co-exist and cannot be easily separated from one another. One risk in this joint management of clinical and administrative issues is that administrative criteria eventually prevail over a sound HTA assessment of the product concerned.

References

[1] Seidman G, Atun R. Do changes to supply chains and procurement processes yield cost savings and improve availability of pharmaceuticals, vaccines or health products? A system-atic review of evidence from low-income and middle-income countries. BMJ Glob Health. 2017 Apr 13;2(2):e000243.

[2] Motile D, De Ponti F, Peruzzi E, Martini N, Rossi P, Silvan MC, Vaccheri A, Montanaro N. An update on the first decade of the European centralized procedure: how many innova-tive drugs? Br J Clin Pharmacol. 2006 Nov;62(5):610-6.

On 2017 Mar 03, Ole Jakob Storebø commented:

In their editorial, Gerlach and colleagues make several critical remarks (Gerlach M, 2017) regarding our Cochrane systematic review on methylphenidate for children and adolescents with attention-deficit hyperactivity disorder (ADHD) (Storebø OJ, 2015). While we thank them for drawing attention to our review we shall here try to explain our findings and standpoints.

They argue, on the behalf of the World Federation of ADHD and EUNETHYDIS, that the findings from our Cochrane systematic review contrast with previously published systematic reviews and meta-analyses, (National Collaborating Centre for Mental Health (UK), 2009, Faraone SV, 2010, King S, 2006, Van der Oord S, 2008) which all judged the included trials more favorably than we did.

There are methodological flaws in most of these reviews that could have led to inaccurate estimates of effect. For example, most of these reviews did not publish an a priori protocol (Faraone SV, 2010, King S, 2006, Van der Oord S, 2008), or present data on spontaneous adverse events (Faraone SV, 2010, King S, 2006, Van der Oord S, 2008), nor did they report on adverse events as measured by rating scales (Faraone SV, 2010, King S, 2006, Van der Oord S, 2008), or systematically assess the risk of random errors, risk of bias, and trial quality (Faraone SV, 2010, King S, 2006,Van der Oord S, 2008). King at al. emphasised in the quality assessments for the NICE review that almost all studies did not score very well in the quality assessments and, consequently, results should be interpreted with caution (King S, 2006).

The authors of this editorial refer to many published critical editorials and they argue that the issues they have raised have not adequately been addressed adequately by us. On closer examination, it is clear that virtually the same criticism has been levelled at us each time by the same group of authors, published in several journal articles, blogs, letters, and comments (Banaschewski T, 2016, BMJ comment,Banaschewski T, 2016, Hoekstra PJ, 2016, Hoekstra PJ, 2016, Romanos M, 2016, Mental Elf blog.

Each time, we have refuted repeatedly with clear counter-arguments, recalculation of data, and detailed explanations (Storebø OJ, 2016, Storebø OJ, 2016,Storebø OJ, 2016, Pubmed commment, Storebø OJ, 2016, BMJ comments, Responses on Mental Elf, Pubmed comment.

Our main point is that the very low quality of the evidence makes it impossible to estimate, with any certainty, what the true magnitude of the effect might be.

It is correct that a post-hoc exclusion of the four trials with co-interventions in both MPH and control groups and the one trial of preschool children changes the standardised mean difference effect size from 0.77 to 0.89. However, even if the effect size increases upon excluding these trials, the overall risk of bias and quality of the evidence deems this discussion irrelevant. As mentioned above, we have responded several times to this group of authors Storebø OJ, 2016, Storebø OJ, 2016,< PMID: 27138912, Pubmed commment, Storebø OJ, 2016, BMJ comments, Responses on Mental Elf, Pubmed comment.

We did not exclude any trials for the use of the cross-over design, as these were included in a separate analysis. The use of end-of-period data in cross-over trials is problematic due to the risk for “carry-over effect” (Cox DJ, 2008) and “unit of analysis errors” (http://www.cochrane-handbook.org). In addition, we tested for the risk of “carry-over effect”, by comparing trials with first period data to trials with end-of-period data in a subgroup analysis. This showed no significant subgroup difference, but this analysis has sparse data and one can therefore not rule out this risk. Even with no statistical difference in our subgroup analysis comparing parallel group trials to end-of-period data in cross-over trials, there was high heterogeneity. This means that the risk of “unit of analysis error” and “carry-over effect” is uncertain, and could be real. The aspect about our bias assessment have been raised earlier by these authors and others affiliated to the EUNETHYDIS. In fact, we see nothing new here. There is considerable evidence that trials sponsored by industry overestimate benefits and underestimate harms (Flacco ME, 2015, Lathyris DN, 2010, Kelly RE Jr, 2006). Moreover, the AMSTAR tool for methodological quality assessment of systematic reviews includes funding and conflicts of interest as a domain (http://amstar.ca/). The Cochrane Bias Methods Group (BMG) is currently working on including vested interests in the upcoming version of the Cochrane Risk of Bias tool.

The aspect about whether teachers can detect well known adverse events of methylphenidate have also been raised earlier by these authors and others affiliated to the EUNETHYDIS (Banaschewski T, 2016, BMJ comment,Banaschewski T, 2016, Hoekstra PJ, 2016, Hoekstra PJ, 2016, Romanos M, 2016, Mental Elf blog.). We have continued to argue that teachers can detect the well-known adverse events of methylphenidate, such as the loss of appetite and disturbed sleep. We highlighted this in our review (Storebø OJ, 2015) and have answered this point in several replies to these authors (Storebø OJ, 2016, Storebø OJ, 2016,Storebø OJ, 2016, Pubmed commment, Storebø OJ, 2016, BMJ comments, Responses on Mental Elf, Pubmed comment. The well-known adverse events of “loss of appetite” and “disturbed sleep” are easily observable by teachers as uneaten food left on lunch plates, yawning, general tiredness, and weight loss.

We have considered the persistent, repeated criticism by these authors seriously, but no evidence was provided to justify changing our conclusions regarding the very low quality of evidence of methylphenidate trials, which makes the true estimate of the methylphenidate effect unknowable. This is a methodological rather than a clinical or philosophical issue.<br> We had no preconceptions of the findings of this review and followed the published protocol; therefore, any proposed manipulations of the data proposed by this group of authors would be in contradiction to the accepted methods of high-quality meta-analyses. As we have repeatedly responded clearly to the criticism of these authors, and it is unlikely that their view of our (transparent) work is going to change, we propose to agree to disagree.

Finally, we do not agree that the recent analysis from registries provides convincing evidence on the long-term benefits of methylphenidate due to multiple limitations of this type of kind of study, albeit that interesting perspectives are provided. They require further study to be regarded as reliable.

Ole Jakob Storebø, Morris Zwi, Helle B. Krogh, Erik Simonsen, Carlos Renato Maia, Christian Gluud

On 2017 Jun 29, Lydia Maniatis commented:

It is interesting that the strawman description (the purely feedforward description) that Heeger is correctly and trivially rejecting in this article is the description serving as the major theoretical premise of a more recent PNAS article (Greenwood, Szinte, Sayim and Cavanagh (2017)) of which Heeger served as editor, and which I have been extensively critiquing. Some representative quotes from that article:

"Given that the receptive fields at each stage in the visual system are likely built via the summation of inputs from the preceding stages..."

"...idiosyncrasies in early retinotopic maps...would be propagated throughout the system and magnified as one moved up the cortical hierarchy."

"Given the hierarchical structure of the visual system, with inherited receptive field properties at each stage..."

These descriptions are never qualified in Greenwood et al (2017), and guide the interpretation of data. How does Heeger reconcile the assertions in the paper he edited with the assertions in his own paper?

On 2017 Jun 17, Lydia Maniatis commented:

Given that the number of possible realities (distribution of matter and light) underlying each instance of retinal stimulation is infinite; given that each instance of retinal stimulation is unique: given that any individual's experience represents only a small subsample of possible experience; given that, as is well-known, lifetime experience (not least for the reasons given above) cannot explain perception (what do we see when we lack the experience needed to re-cognize something?), Heeger's (2017) references to prior probability distributions are unintelligible (and thus untestable).

The question of how these statistical distributions are supposed to be instantiated in the brain is also left open, another reason this non-credible "theory" is untestable. All we have is a set of equations that can't be linked to any relevant aspect of the reality they're supposed to explain.

On 2017 Mar 10, Stuart RAY commented:

Response from EASL here: EASL Recommendations for Treatment of Hepatitis C 2016 Panel. Electronic address: easloffice@easloffice.eu., 2017 [not linked above in PubMed, currently]

On 2017 Jun 12, Randi Goldman commented:

I'm the first author on this study, and there's now a free online version of this tool that can be used for counseling patients. I hope you find it useful: https://www.mdcalc.com/bwh-egg-freezing-counseling-tool-efct

On 2017 Jul 05, Anne Niknejad commented:

error in figure3 legend: "The enzyme activity with pNP-myristate was taken as 100%."

should be "The enzyme activity with pNP-butyrate was taken as 100%."

to be in accordance with results displayed and text: "Maximum enzymatic activity was observed with pNP-butyrate (C4) (100%, 176.7U/mg), while it showed weaker activity with p-NP acetate (C2, 53.08%), p-NP octanoate (C8, 25.59%), p-NP deconoate (C10, 35.05%), p-NP laureate (C12, 18.51%), p-NP myristate (C14, 7.39%), p-NP palmitate (C16, 2.23%) than that with C4 ( Fig. 3)."

Also, it is 'decanoate', not 'deconoate' (this kind of details impacts on text mining)

On 2017 Apr 03, Randi Pechacek commented:

One of the authors of this paper, Despina Lymperopoulou, wrote about this paper on microBEnet discussing some of the background. Read about it here.

On 2017 Feb 14, Peter Hajek commented:

To clarify the dependence potential of vaping, could the authors provide data on never-smokers who became daily vapers please?

On 2017 May 15, Lydia Maniatis commented:

In short, there are too many layers of uncertainty and conceptual vagueness here for this project to offer any points of support for any hypothesis or theory.

On 2017 May 15, Lydia Maniatis commented:

Every hypothesis or theory is a story, but in the current relaxed climate in vision science, the story doesn't need to be empirically tested and well-rationalized in order to be publishable. We just need a special section in the paper acknowledging these problems, often entitled "limitations of the study" or, as here, "qualifying remarks." Below I excerpt remarks from this and other sections of the paper (caps mine):

"A major goal of the present work was to test hypotheses related to multi-stage programming of saccades in infants. On the empirical side, we exposed 6-month-old infants to double-step trials but DID NOT SUCCEED IN COLLECTING RELIABLE DATA. In this respect, the model simulations were used to investigate an aspect of eye movement control that was not tested empirically."

"In the model simulations, certain model parameters were allowed to vary across age groups and/or viewing conditions, based on theoretical and empirical considerations. We then interpreted the constellation of best-fitting parameters." There is a great deal of flexibility in post hoc data-fitting with numerous free parameters.

"in supplementary analyses (not presented here) we used this approach to follow up on the results obtained in Simulation Study 2. To determine the individual contributions of saccade programming and saccade timing model parameters in generating the fixation duration distributions from LongD and ShortD groups during free viewing of naturalistic videos, we ran simulations in which we estimated the saccade programming parameters (mean durations of labile and non-labile stages) while keeping the saccade timing parameters fixed, and vice versa. In brief, the results confirmed that, for both ShortD and LongD groups, a particular combination of saccade-programming and saccade timing parameters was needed to achieve a good fit. HOLDING EITHER SET OF PARAMETERS FIXED DID NOT RESULT IN AN ADEQUATE FIT."

There are also a lot of researcher degrees of freedom in generating and analysing data. From the methods:

"Fixation durations (FDs). Eye-tracking data from infants may contain considerably higher levels of noise than data from more compliant participants such as adults due to various factors including their high degree of movement, lack of compliance tothe task, poor calibration and corneal reflection disturbances dueto the underdeveloped cornea and iris (Hessels, Andersson,Hooge, Nystr歬 & Kemner, 2015; Saez de Urabain, Johnson, &Smith, 2015; Wass, Smith, & Johnson, 2013). To account for this potential quality/age confound, dedicated in-house software for parsing and cleaning eye tracking data has been developed (GraFix, Saez de Urabain et al., 2015). This software allows valid fixations to be salvaged from low-quality datasets whilst also removing spurious invalid fixations. In the present study, both adult and infant datasets were parsed using GraFix鳠two-stage semi-automated process (see Appendix A for details). The second stage of GraFix involves manual checking of the fixations detected automatically during the first stage. This manual coding stage was validated by assessing the degree of agreement between two different raters. ONE RATER WAS ONE OF THE AUTHORS (IRSdU)."

The "in-house software" changes the data, a d therefore the assumptions implicit in its computations should be made explicit. The p-value used to assess rater agreement was p<05, which nowadays is considered rather low. We need more info on the "valid/invalid" distinction as well as on how the software is supposed to make this distinction.

From the "simulation studies": "The parameter for the standard deviation of the gamma distributions (rc) is a fixed parameter. To accommodate the higher variability generally observed in infant data compared to adult data, it was set to 0.33 for the infant data and 0.25 for the adult data. These values were adopted from previous model simulations (Engbert et al., 2005; Nuthmann et al., 2010; Reichle et al., 1998, 2003)."

Using second-hand values adopted by other researchers in the past doesn't absolve the current ones from explaining the rationale behind these choices (assuming there is one.) "

On 2017 Feb 07, Dejan Stevanovic commented:

Since this review was done in 2014, one study has been published supporting the cross-cultural validity of the Revised Child Anxiety and Depression Scale (RCADS) (https://www.ncbi.nlm.nih.gov/pubmed/27353487).

Two studies have been published evidencing that the self-report Strengths and Difficulties Questionnaire (SDQ) lacks corss-cultural validity and it is not suitable for cross-cultural comparisons (https://www.ncbi.nlm.nih.gov/pubmed/28112065, https://www.ncbi.nlm.nih.gov/pubmed/?term=New+evidence+of+factor+structure+and+measurement+invariance+of+the+SDQ+across+five+European+nations).

On 2017 Sep 19, Helmi BEN SAAD commented:

The exact names of the authors are: Ben Moussa S, Sfaxi I, Ben Saad H, Rouatbi S".

On 2017 Sep 19, Helmi BEN SAAD commented:

The exact names of the authors are: Ben Saad H, Khemiss M, Nhari S, Ben Essghaier M, Rouatbi S".

On 2017 Feb 13, Khaled Moustafa commented:

Thank you, Josip, for your thoughtful comments. It is a general issue, indeed, among many others that need to be fixed in the publishing industry. Hopefully, some publishers will be all ears.

Regards,

KM

On 2017 Feb 08, Josip A. Borovac commented:

Great input, Khaled! Thank you so much for these observations. You are not alone my friend! Best wishes, JAB

On 2017 Feb 08, Khaled Moustafa commented:

Thank you all for your comments.

Further reasons that strongly support the idea not to ask for any specific format or style at the submission stage include:

1) It is rare that a manuscript is accepted immediately from the first run; in most cases there is a revision, nonetheless a minor one. So, editors can ask authors to apply the journal’s style and page setup in the revised version only, but not at the submission stage.

2) In most cases, the final publication format is different from the initial submission format, whatever the style required by the journal and applied by the authors. That is, even if we apply a particular journal's style at the submission stage, the accepted final version (i.e., the PDF file) often appears in a different format and style than the initial one required at the submission stage. So, asking for a given page setup, citation style or specific format at the submission phase but not taking it into account in the final version is an obvious waste of authors’ time. Much time indeed is lost in doing things that are not taken into consideration in the final published versions (except maybe in the HTML version or authors’ version posted online prior to the proof version but not in the final PDF format. So, once again, it does not make much sense to require a drastic formatting at the submission stage).<br> Regardless of innumerable references’ styles (by name or date, with superscript or brackets, journal names in italic or not, in bold or not, underlined or not, etc.), some journals return manuscripts to the author just because the references are non-indented or indented… or because the headers were enumerated/non-enumerated... Other journals ask to upload two files of the same manuscript (a Word file and a PDF file), some others ask to include the images/figures or tables in the text or in separated files…etc. All these are trivial issues related to the form that does not change the inherent value of a manuscript. As it is the content that should matter but not the format or style, page setup or styles could be done only in a revised version when the manuscript is accepted but not elsewhere.<br> At least, journals should make it optional for authors to apply or not to apply the journal’s style at the submission stage. On another hand, the submission steps, in turn, are also long and overwhelming in many journals. In my view, these also need to be shortened to the strict minimum (for .e.g., login and upload files). Then, if the manuscript is accepted, authors could provide the long list of the required information (statement of conflict of interest, list of keywords, all the answers and questions currently stuffing in the submission process, etc.).

Regards,

KM

On 2017 Feb 07, Saul Shiffman commented:

Super-agree. Even application of this attitude to paper length could be useful. I'm sure we've all gone to the trouble of whittling a paper down to the (ridiculously low) word-count requirements of a particular journal, only to have the paper rejected.

On 2017 Feb 07, Josip A. Borovac commented:

Agree with this - a well-identified problem and good article. Some journals are trying to implement "your paper - your way" style, but this should be taken to a whole another level, generally speaking. Many journals sponsor very obscure formatting styles and to resubmit to another journal is a nightmare and definitely time-consuming. Researchers should focus on science as much as possible, much less on submission technicalities and crude formatting issues. I never understood a point of having 3 billion citation styles, for example. What is the true purpose of that except making our lives miserable?

On 2017 Feb 07, Thittayil Suresh Apoorv commented:

This is nice suggestion by the author. Some journals already following this. Journal like Cytokine are part of Elsevier’s Article Transfer Service (ATS). Reformatting is required only after acceptance.

On 2017 Feb 07, Francesco Brigo commented:

I couldn´t agree more: time is brain!

On 2017 May 10, Christopher Tench commented:

Implementation errors in the GingerALE Software: Description and recommendations http://onlinelibrary.wiley.com/doi/10.1002/hbm.23342/full

On 2017 Apr 13, Konstantinos Fountoulakis commented:

This paper discusses possible mechanisms of antidepressant effect 1. Although the whole discussion is intriguing, it should be noted that the fundamental assumption of the authors is mistaken. More specifically, the authors start from the position that there is a latency time of approximately two weeks from initiation of antidepressant treatment to the manifestation of the treatment effect. This is an old concept, now proven wrong. We now know that the treatment response starts within days and it takes two weeks not for the treatment effect to appear but for the medication group to separate from placebo 2. This conclusion is so solid that it has been incorporated in the NICE CG90 guidelines for the treatment of depression (available at https://www.nice.org.uk/guidance/cg90/evidence/full-guidance-243833293, page 413). These two are completely different concepts, often confounded in the literature. However it is clear that medication significantly improves the chances of a patient to be better after two weeks in comparison to placebo, but improvement itself has started much earlier. We also know that the trajectories of patients we respond to medication are similar to the trajectories of those who improve under placebo. One of the possible consequences of this observation is that there might not be a physiological difference underlying response under medication in comparison to response under placebo; however the chances these physiological mechanisms are activated are higher under medication References 1. Harmer CJ, Duman RS, Cowen PJ. How do antidepressants work? New perspectives for refining future treatment approaches. The lancet Psychiatry 2017. 2. Posternak MA, Zimmerman M. Is there a delay in the antidepressant effect? A meta-analysis. The Journal of clinical psychiatry 2005; 66(2): 148-58.

On 2017 Sep 19, NephJC - Nephrology Journal Club commented:

This trial of extending hemodialysis hours and quality of life was discussed on September 12th and September 13th 2017 on #NephJC, the open online nephrology journal club. Introductory comments written by Swapnil Hiremath are available at the NephJC website here .

The highlights of the tweetchat were:

• The study was well-designed and relevant.

• Interpreting the results, can you validly fit a linear model to a questionnaire score?

• Some would argue that including both incident and prevalent patients may have confounded some of the results as LV mass regresses in the first few months after initiation.

• This paper confirmed previous literature that preserving residual renal function is really essential for better outcomes on dialysis and that HD dose should track with this.

• Fluid control may be more important than solute clearance.

Transcripts of the tweetchats, and curated versions as storify will be shortly available from the NephJC website.

Interested individuals can track and join in the conversation by following @NephJC or #NephJC on twitter, liking @NephJC on facebook, signing up for the mailing list, or just visit the webpage at NephJC.com.

On 2017 Feb 06, Clive Bates commented:

To build on to Professor Hajek's cogent criticism, I would like to add a further three points:

First, the authors offer the usual disclaimer that they cannot make causal inferences from a study of this nature, which is correct. But they go on to do exactly that within the same paragraph:

Finally, although the crosssectional design of our study allowed us to determine associations between variables, it restricted our ability to draw definitive causal inferences, particularly about the association between ENDS use and smoking cessation. Nevertheless, the association between ENDS use and attempts at smoking cessation suggests that a substantial proportion of smokers believe that ENDS use will help with smoking cessation. Furthermore, the inverse association between ENDS use and smoking cessation suggests that ENDS use may actually lower the likelihood of smoking cessation. (emphasis added)

From that, they build a policy recommendation:

"Tobacco cessation programs should tell cigarette smokers that ENDS use may not help them quit smoking"

That statement is literally true for e-cigarettes and every other way of quitting smoking, but it is not a meaningful or legitimate conclusion to draw from this study because the design does not allow for causal inferences.

Second, the authors characterise e-cigarette use as 'ever use' in calculating their headline odds ratio (0.53).

Our most important finding was that having ever used ENDS was significantly associated with reduced odds of quitting smoking.

Ever use could mean anything from 'used once and never again' to 'use all day, every day' or 'used once when I couldn't smoke'. What it does not mean is 'used an e-cigarette in an attempt to quit smoking'. So this way of characterising e-cigarette use can tell us little about people who do try to quit smoking using an e-cigarette or whether that approach should be recommended.

Third, as well as the basic timing point made by Professor Hajek, the authors do not consider a further obvious contributory explanation: reverse causality. It is quite possible that those who find it hardest to quit or don't want to quit may be those who are drawn to trying e-cigarettes - either because they don't want to stop or have tried everything else already and failed. It is not safe to assume that the population is homogeneous in the degree of nicotine dependence, that e-cigarette ever-use is randomly distributed across the population or that e-cigarette use is generally undertaken with the intention of quitting.

The analysis provides no insights relevant to the efficacy of e-cigarettes in smoking cessation and building any sort of recommendation to smoking cessation programs based on this survey is wrong and inappropriate.

On 2017 Feb 06, Peter Hajek commented:

The unsurprising finding that people who quit smoking between 2009 and 2013 were less likely to try e-cigarettes than those who still smoked in 2014 is presented as if this shows that the experience with vaping somehow ‘reduced odds of quitting smoking’. It shows no such thing.

It is obvious that current smokers must be more likely to try e-cigarettes than smokers who quit years ago. E-cigarettes were more widely used in 2014 than in previous years. People who quit smoking up to five years earlier would have much less (or even no) opportunities to try vaping, and no reason to do so after they quit. Current smokers in contrast continue to have a good reason to try e-cigarettes, and are having many more opportunities to do so. This provides no information at all about odds of quitting smoking or about whether e-cigarettes are effective or not.

On 2017 Apr 12, Lydia Maniatis commented:

It occurs to me that the there are two ways to interpret the finding that people were influenced by the stable versions of the dress image in the different settings. The authors say that these versions introduced a bias as to the illumination. But it seems to me more straightforward to assume that they introduced a bias or expectation with respect to the actual colors of the dress, that is a perceptual set mediating the latter. It took me a while to realize this - an example of how explanations that are given to us can create a 'conceptual set.'

On 2017 Feb 15, Lydia Maniatis commented:

I don’t think the authors have addressed the unusual aspects of the dress in saying that “The perceived colors of the dress are due to (implicit) assumptions about the illumination.” As they note themselves, “This is exactly what would be predicted from classical color science…”

The spectrum of light reflected to our eye is a function of the reflectance properties of surfaces and the spectrum of the illumination; both are disambiguated on the basis of implicit assumptions, and both are represented in the percept. The two perceptual features (surface color and illumination) are two sides of the same coin: Just as we can say that seeing a surface as having color x of intensity y is due to assumptions about the color and intensity of the illuminants, so we can say that seeing illumination of color x and intensity y is due to implicit assumptions about the reflectance (how much light they reflect) and the chromaticity (which wavelengths they reflect/absorb) of the viewed surfaces. We haven’t explained anything unless we can explain both things at the same time.

The authors are choosing one side of the perceptual coin – the apparent illumination – and claiming to have explained the other. Again, it’s a truism to say that seeing a patch of the dress as color “x” implies we are seeing it as being under illumination “y,” while perceiving the patch as a different color means perceiving a different illumination. This doesn’t explain what makes the dress unusual - why it produces different color/illumination impressions in different people.

The authors seem to want to take the “experience” route (“prior experiences may influence this perception”); this is logically and empirically untenable, as has been shown and argued innumerable times in the vision literature. For one thing, such a view is circular, since what we see in the first place is a product of the assumptions implicit in the visual process. It’s not as though we see things first, and then adopt assumptions that allow us to see it…In addition, why would such putative experience influence only the dress, and not each and every percept? (The same objection applies to explanations in terms of physiological differences). Again, the question of what makes the dress special is left unaddressed.

It’s odd that, for another example of such a phenomenon, vision researchers need to turn to “poppunkblogger.” If they understood it in principle, then they would be able to construct any number of alternative versions. Even if they could show the perception of the dress to be experience-based (which, again, is highly unlikely to impossible), this would not not help; they would still be at a loss to explain why different people see different versions of one image and not most others. To understand the special power of the dress, they need at a minimum to analyze its structure, not only in terms of color but in terms of shape, which is the primary mediator of all aspects of perception. Invoking “scene interpretation” and “the particular color distributions” are only placeholders for all the things the authors don’t understand.

The construction of images that show that the dress itself can produce consistent percepts is genuinely interesting, but it is a problem that the immediate backgrounds are not the same (e.g. arm placements). This produces confounds. The claim that these confounds are designed to produce the opposite effect of what is seen, based on contrast effects, is not convincing, since the idea that illusions involving transparency/illumination are based on local contrast effects is a claim that is easy to falsify empirically, and has been falsified. So we are dealing with unanalyzed confounds, and one has to wonder how much blind trial and error was involved in generating the images.

Finally, I’m wondering why a cutout of the dress wasn’t also placed against a plain background as a control; what happens in this case? Has this been done yet?

On 2017 Nov 03, Elisabeth Schramm commented:

In reply to a comment by Falk Leichsenring

Allegiance effects controlled

Elisabeth Schramm, PhD; Levente Kriston, PhD; Ingo Zobel, PhD; Josef Bailer, PhD; Katrin Wambach, PhD; Matthias Backenstrass, PhD; Jan Philipp Klein, MD; Dieter Schoepf, MD; Knut Schnell, MD; Antje Gumz, MD; Paul Bausch, MSc; Thomas Fangmeier, PhD; Ramona Meister, MSc; Mathias Berger, MD; Martin Hautzinger, PhD; Martin Härter,MD, PhD

Corresponding Author: Elisabeth Schramm, PhD, Department of Psychiatry, Faculty of Medicine, University of Freiburg, Hauptstrasse 5, 79104 Freiburg, Germany (elisabeth.schramm@uniklinik-freiburg.de)

We acknowledge the comment of Drs. Steinert and Leichsenring (1) on our study (2) reasoning that our findings may at least in part be attributed to allegiance effects. Unfortunately, they provide neither a clarification of what they exactly refer to with the term “allegiance effects” nor a specific description of the presumed mechanisms (chain of effects) through which they think allegiance may have influenced our results. In fact, as specified both in the trial protocol (3) and the study report (2), we took a series of carefully safeguarded measures to minimize bias. Unlike stated in the comment, training and supervision of the study therapists and the center supervisors were performed by qualified and renowned experts for both investigated approaches (Martin Hautzinger for Supportive Psychotherapy and Elisabeth Schramm for Cognitive Behavioral Analysis System of Psychotherapy). Moreover, none of them has been involved in treating any study patients in this trial. We are confident that any possible allegiance of the participating researchers, therapists, supervisors, or other involved staff towards any, both, or none of the investigated interventions is very unlikely to have been able to surmount all of the implemented measures against bias and to affect the results substantially.

References

(1) Steinert C, Leichsenring F. The need to control for allegiance effects in psychotherapy research. PubMed Commons. Sep 08 2017

(2) Schramm E, Kriston L, Zobel I, Bailer J, Wambach K, Backenstrass M, Klein JP, Schoepf D, Schnell K, Gumz A, Bausch P, Fangmeier T, Meister R, Berger M, Hautzinger M, Härter M. Effect of Disorder-Specific vs Nonspecific Psychotherapy for Chronic Depression: A Randomized Clinical Trial. JAMA Psychiatry. Mar 01 2017; 74(3): 233-242

(3) Schramm E, Hautzinger M, Zobel I, Kriston L, Berger M, Härter M. Comparative efficacy of the Cognitive Behavioral Analysis System of Psychotherapy versus supportive psychotherapy for early onset chronic depression: design and rationale of a multisite randomized controlled trial. BMC Psychiatry. 2011;11:134

On 2017 Sep 08, Falk Leichsenring commented:

The need to control for allegiance effects in psychotherapy research

Christiane Steinert1,2, PhD and Falk Leichsenring, DSc2

1 Medical School Berlin, Department of Psychology, Calandrellistraße 1-9, 12247 Berlin

2 University of Giessen, Department of Psychosomatics and Psychotherapy, Ludwigstraße 76, 35392 Giessen

Corresponding author: Prof. Dr. Falk Leichsenring University of Giessen Department of Psychosomatics and Psychotherapy Ludwigstr. 76, 35392 Giessen, Germany Fon | +49-641-99 45647 Fax | +49-641-99 45669 Mail | falk-leichsenring@psycho.med.uni-giessen.de

In a recent trial on psychotherapeutic efficacy Schramm et al. hypothesized that Cognitive Behavioral Analysis System of Psychotherapy (CBASP) would be superior to supportive therapy (SP) in the treatment of chronic depression.1 This hypothesis was corroborated, CBASP improved depression significantly more, but only by a moderate effect size of 0.31. Some issues, however, raise the question of possible allegiance effects on several levels.2 (1) The authors clearly are in favour of CBASP and expect it to be superior to SP (primary hypothesis). (2) Furthermore, six authors participated as therapists in the CBASP group, while none of the authors seems to have participated in the SP group. The authors can be expected to be alleged to CBASP. (3) In addition, the therapy sessions were supervised by the very same authors participating in the CBASP group as therapists. (4) No expert of SP was listed to have participated in the study, neither as a researcher, a therapist nor as a supervisor. (5) The authors stated that all therapists completed a 3-year psychotherapeutic training program or were in an advanced stage of training. However, at least in Germany, there is no 3-year training program specifically for SP, only for CBT. Thus, it is unlikely that the therapists in the SP condition were alleged to SP in the same way as the therapists in the CBASP condition. Further information on the background of the therapists in SP would be informative.

Thus, a researcher, a therapist and a supervisor allegiance effect can be expected to be present in this study.3

Munder et al. found an association between researcher allegiance and outcome of r=0.35 which corresponds to a medium effect size.2 For this reason the possibility cannot be ruled out that the moderate between-group effect size of 0.31 is at least in part due to allegiance effects. - The fact that the treatments do not seem to differ with regard to treatment fidelity ratings does not rule out this possibility. This is also true for the fact that therapists met the criteria for mastery of CBASP and SP before treating study patients.

References

1. Schramm E, Kriston L, Zobel I, Bailer J, Wambach K, et al. Effect of Disorder-Specific vs Nonspecific Psychotherapy for Chronic Depression: A Randomized Clinical Trial. JAMA Psychiatry. Mar 01 2017;74(3):233-242.
2. Munder T, Flückiger, C, Gerger, H, Wampold, BE, Barth, J. Is the Allegiance Effect an Epiphenomenon of True Efficacy Differences Between Treatments? A Meta-Analysis. J Couns Psychol. 2012(Epub ahead of print).
3. Steinert C, Munder T, Rabung S, Hoyer J, Leichsenring F. Psychodynamic Therapy: As Efficacious as Other Empirically Supported Treatments? A Meta-Analysis Testing Equivalence of Outcomes. Am J Psychiatry. May 25 2017:appiajp201717010057.
4. Cuijpers P, Huibers MJ, Furukawa TA. The Need for Research on Treatments of Chronic Depression. JAMA Psychiatry. Mar 01 2017;74(3):242-243.

On 2017 Feb 16, Lydia Maniatis commented:

"Our findings suggest that people who perceive the dress as blue might rely less on contextual cues when estimating surface color."

Can there be a "context-free" condition, in principle? What would this look like? The term context seems far too general as it is used here. The conclusion as framed has no theoretical content. If the reference is to specific manipulations, and the principles behind them, then it's an entirely different thing, and should be specified.

The fact that the results of this study differed significantly from the results of others should be of concern with respect to all of them. Might replication attempts be in order, or are chatty post-mortems enough?

"Our results lend direct support to the idea that blue and white perceivers see the dress in a different color because they discount different illumination colors."

This statement involves a major conceptual error in the sense that it cannot function as an explanation. The visual system infers both light and illumination from the stimulation of the retina by various wavelengths of various intensities. Both surface appearance and illumination are inferred from the same stimulation; to make an inference about illumination is to simultaneously make an inference about reflectance/chromaticity. One “explains” the other in the sense that each inference is contingent on the other; but to say that one inference has priority over the other is like saying the height of one side of a see-saw determines the height of the other; it’s an empty statement. What we need to explain is the movement of the whole, interconnected see saw.

This error is unfortunately a common one; it's also made by Witzel, Racey and O'Regan (2017) in this special issue. In short, this is a non-explanation.

On 2017 Feb 08, Raphael Stricker commented:

Another Lyme OspA Vaccine Whitewash

The meta-analysis by Zhao and colleagues comes to the conclusion that "the OspA vaccine against Lyme disease is safe and its immunogenicity and efficacy have been verified." The authors arrive at this sunny conclusion by excluding 99.6% of published articles that demonstrate potential problems with the OspA vaccine. Furthermore, the authors ignore peer-reviewed studies, FDA regulatory meetings and legal proceedings that point to major problems with OspA vaccine safety (1-3). This whitewash bodes ill for future Lyme vaccine candidates because it fosters disregard for vaccine safety among Lyme vaccine manufacturers and mistrust among potential Lyme vaccinees.

References 1. Stricker RB (2008) Lymerix® risks revisited. Microbe 3: 1–2. 2. Marks DH (2011) Neurological complications of vaccination with outer surface protein A (OspA). Int J Risk Saf Med. 23: 89–96. 3. Stricker RB, Johnson L (2014) Lyme disease vaccination: safety first. Lancet Infect Dis. 14(1):12.

Disclosure: RBS is a member of the International Lyme and Associated Diseases Society (ILADS) and a director of LymeDisease.org. He has no financial or other conflicts to declare.

On 2017 Feb 02, Stuart RAY commented:

Regarding the prominent concluding statement of the abstract, what is the evidence that treatment success (with current recommended regimens) will be reduced by the RASs found in this study?

On 2017 Feb 15, thomas samaras commented:

This is an excellent paper on height trends in Sardinia. However, I disagree with the precept that height can be used as a measure of health and longevity. Many researchers view height as a byproduct of the Industrial Revolution and the Western diet. In actuality, greater height and associated weight is harmful to our long-term health and longevity. There are many reasons for this position.

1. Carrera-Bastos reported that our modern diet is not the cause for increased life expectancy (LE). Instead, our progress in sanitation, hygiene, immunization, and medical technology have driven our rise in life expectancy. This increase in life expectancy is not that great at older ages; e.g., in 1900, a 75-year old man could expect to live another 8.5 years. A 100 years later, he could expect to live 10 years. Not a substantial improvement in spite of great advances in food availability, lifestyle, medicine, worker safety, etc.

2. A number of researchers have associated our increased height with excess nutrition, not better quality nutrition (Farb, Galton, Gubner, and Campbell).

3. Nobel prize winner, Charles Townes stated that shorter people live   
   

   longer. Other scientists supporting the longevity benefits of smaller<br> body size within the same species include Bartke, Rollo, Kraus, Pavard, Promislow, Richardson, Topol, Ringsby, Barrett, Storms, Moore, Elrick, De Magahlaes and Leroi.

4. Carrera-Bastos, et al. reported that pre-Western societies rarely get age-related chronic diseases until they transition to a Western diet. Trowell and Burkitt found this to be true based on their research over 40 years ago (Book: Western Diseases, Trowell and Burkitt.) Popkin noted that the food system developed in the West over the last 100+ years has been “devastating” to our health.

5. A 2007 report by the World Cancer Research Fund/American Institute of Cancer Research concluded that the Industrial Revolution gave rise to the Western diet that is related to increased height, weight and chronic diseases. (This report was based on evaluation of about 7000 papers and reports.)

6. US males are 9% taller and have a 9% shorter life expectancy. Similar differences among males and females in Japan and California Asians were found. It is unlikely that the inverse relationship in life expectancy and height is a coincidence. (Bulletin of the World Health Organization, 1992, Table 4.)

7. High animal protein is a key aspect of the Western Diet, but it has many negative results. For example, a high protein diet increases the levels of CRP, fibrinogen, Lp (a), IGF-1, Apo B, homocysteine, type 2 diabetes, and free radicals. In addition, the metabolism of protein has more harmful byproducts; e.g., the metabolism of fats and carbs produces CO2 and water. In contrast protein metabolism produces ammonia, urea, uric acid and hippuric acid. (Fleming, Levine, Lopez).

8. The high LE ranking of tall countries is often cited as supporting the the conviction that taller people live longer. However, if we eliminate non-developed countries, which have high death rates during the first 5 years of life and poor medical care, the situation changes. However, among developed countries, shorter countries rank the highest compared to tall countries. For example, out of the top 10 countries, only Iceland is a tall country. The other developed countries are relatively short or medium in height: The top 10 countries include: Monaco (1), Singapore, Japan, Macau, San Marino, Iceland (tall exception), Hong Kong, Andorra, Switzerland, and Guernsey (10). The Netherlands, one of the tallest countries in Europe, ranks 25 from the top. The ranking of other tall countries include: Norway (21), Germany (34), Denmark (47), and Bosnia and Herzegovina (84). Source for LE data: CIA World Factbook, 2016 data. Male height data from Wikipedia.

It should be pointed out that a number of confounders exist that can invalidate mortality studies that show shorter people have higher mortality. Some of these confounders include socioeconomic status, higher weight for height in shorter people, smoking, and failure to focus on ages exceeding 60 years (differences showing shorter people live longer generally occur after 60 years of age). For example, Waaler’s mortality study covered the entire age range. He found that between 70 and 85 years of age, tall people had a higher mortality than shorter men between 5’7” and 6’. An insurance study (Build Study, 1979) found that when they compared shorter and taller men with the same degree of overweight, the shorter men had a slightly lower mortality.

Anyone interested in the evidence showing that smaller body size is related to improved health and longevity can find evidence in the article below which is based on over 140 longevity, mortality, survival and centenarian studies.

Samaras TT. Evidence from eight different types of studies showing that smaller body size is related to greater longevity. JSRR 2014. 2(16): 2150-2160, 2014; Article no. JSRR.2014.16.003

On 2017 Feb 01, Hilda Bastian commented:

The authors raise interesting and important points about the quandaries and complexities involved in updating a systematic review and reporting the update. However, their review of the field and conclusion that of the 250 journals they looked at, only BMC Systematic Reviews has guidance on the process of updating is deeply flawed.

One of the 185 journals in the original sample they included (Page MJ, 2016) is the Cochrane Database of Systematic Reviews. Section 3.4 of the Cochrane Handbook is devoted to updating, and updating is addressed within several other sections as well. The authors here refer to discussion of updating in Cochrane's MECIR standards. Even though this does not completely cover Cochrane's guidance to authors, it contradicts the authors' conclusion that BMC Systematic Reviews is the only journal with guidance on updating.

The authors cite a recent useful analysis of guidance on updating systematic reviews (Garner P, 2016). Readers who are interested in this topic could also consider the broader systematic review community and methodological guidance. Garritty C, 2010 found 35 organizations that have policy documents at least on updating, and many of these have extensive methodological guidance, for example AHRQ (Tsertsvadze A, 2008). Recently, guidelines for updating clinical guidelines have also been published (Vernooij RW, 2017).

The authors reference some studies that address updating strategies, however this literature is quite extensive. You can use this filter in PubMed along with other search terms for studies and guidance: sysrev_methods [sb] (example). (An explanation of this filter is on the PubMed Health blog.)

Disclosure: I work on PubMed Health, the PubMed resource on systematic reviews and information based on them.

On 2017 Jul 13, GARRET STUBER commented:

The corrected version of this manuscript is now online at the journal's website. A detailed correction notice is linked to the corrected article.

On 2017 Feb 09, GARRET STUBER commented:

None

On 2017 Nov 20, Erin Frazee Barreto commented:

The Cystatin C-Guided Vancomycin Dosing tool can be accessed using the mobile or web app 'Calculate' available at QxMD

https://qxmd.com/calculate/calculator_449/vancomycin-dosing-based-on-egfr-creatinine-cystatin-c

On 2018 Jan 08, Israel Hanukoglu commented:

A three dimensional (3D) video of the human eccrine sweat gland duct covered with sodium channels can be seen at: https://www.youtube.com/watch?v=JcddOILffOM

The video was generated based on the results of this study.

On 2017 May 17, Israel Hanukoglu commented:

Our lab has undertaken to map the sites of expression and localization of ENaC and CFTR in epithelial tissues. This article is the second one in the series and it concentrates on the skin.

In the first paper, we covered the sites of localization of ENaC and CFTR in the respiratory tract and the female reproductive tract. Both of these tissues contain large stretches of epithelium covered with multi-ciliated cells. We had shown that in these epithelia with motile cilia, ENaC is expressed along the entire length of the cilia. Reference: https://www.ncbi.nlm.nih.gov/pubmed/22207244

In the current work on the skin, epidermis and epidermal appendages, ENaC was found mostly located in the cytoplasm of keratinocytes, sebaceous glands, and smooth muscle cells. Only in the eccrine type sweat glands, ENaC and CFTR were found predominantly on the luminal membrane facing the lumen of the ducts. Thus, the reuptake of Na+ ions secreted in sweat probably takes place in the eccrine glands.

On 2017 Jan 28, Eric Fauman commented:

For a report on Somascan pQTLs from a much larger but cross-sectional population check out the recent (not yet peer-reviewed) BioRxiv paper from Karsten Suhre and colleagues:

Connecting genetic risk to disease endpoints through the human blood plasma proteome http://biorxiv.org/content/early/2016/11/09/086793

For example, the association reported above (rs3197999, pvalue=6e-10 for MST1 levels) is reported in the BioRxiv paper with a p-value of 1e-242.

The data from the BioRxiv paper can be explored at http://proteomics.gwas.eu

On 2018 Jan 05, Martine Crasnier-Mednansky commented:

All data in this paper should be discarded simply because the strains used by the authors are not what the authors think they are, as explained below.

An Escherichia coli strain lacking PEP synthase does not grow on pyruvate. In fact, the crucial role of PEP synthase during growth on pyruvate is well documented. In brief, mutant strains were isolated which could grow on glucose or acetate but not on pyruvate; it was found they lacked PEP synthase (see Cooper RA, 1967 for an early paper). Furthermore, because the PEP synthase gene (ppsA) is transcriptionally positively regulated by the fructose repressor FruR (Geerse RH, 1986, also known as Cra), fruR mutant strains are routinely checked for their inability to grow on pyruvate. Therefore, data (in supplementary Fig. 1) indicating wild type and ppsA strains grow equally well on pyruvate are incorrect; the strain used by the authors is not a ppsA strain.

The ptsI strain also does not appear to be a ptsI strain, as it grows on xylose as well as a wild type strain (figure 3b), it should not; growth on xylose requires cAMP, which requires the phosphorylated form of Enzyme IIA^Glc.

On 2017 Mar 12, Atanas G. Atanasov commented:

Silymarin is indeed a very prominent herbal product with a variety of demonstrated bioactivities. It was also recently studied in my group in the context of regulation of PPARgamma activity and macrophage cholesterol efflux. I have enjoyed reading this review focused on the usefulness of the plant product in chronic liver disease, and have featured it on: http://healthandscienceportal.blogspot.com/2017/03/how-beneficial-is-silymarinsilybin-use.html

On 2017 Feb 15, Vojtech Huser commented:

This is a very interesting study. Since it using OMOP CDM, it would be interesting to execute it on additional datasets. The appendix provides some guidance. Are there plans to release (possibly with some delay) additional analysis code details? (the study is possibly using existing software packages where case studies in their application are very valuable)

On 2017 Nov 28, Natalie Clairoux commented:

Free version of this article available at https://papyrus.bib.umontreal.ca/xmlui/handle/1866/19505

On 2017 Feb 22, KEVIN BLACK commented:

The version of record is available via the DOI shown above. The peer-reviewed manuscript version appears by agreement with the publisher at http://works.bepress.com/kjb/66/ .

On 2017 Jan 28, Thomas Jeanne commented:

There is ambiguity in the wording that the authors use in the body text and the abstract to describe the HbA1c reductions that were observed. A 1.1% mean reduction implies a relative reduction; e.g., a reduction from 7.6% A1c to 7.52% A1c. It is only after looking at Table 2 that it becomes clear that the observed change was an absolute reduction in the percentage of hemoglobin that was glycated. To avoid such confusion, use of the term "percentage point" is well-established (e.g., Hayward RA, 1997, Vijan S, 2014, Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications (DCCT/EDIC) Research Group., 2016).

The mean reduction in HbA1c level from baseline was 1.1 percentage points in the CGM group at 12 weeks, not 1.1%.

On 2017 Feb 08, Christophe Dessimoz commented:

Story behind the paper in this blog post: http://lab.dessimoz.org/blog/2017/02/08/sex-alcohol-and-structural-variants-in-fission-yeast

On 2017 Feb 04, Carl V Phillips commented:

None

On 2017 Jan 25, Peter Hajek commented:

The press release claimed that ‘E-Cigarettes are Expanding Tobacco Product Use Among Youth’ but this study showed no such thing. It detected no increase in youth smoking, on the contrary, the continuous decline in smoking shows that e-cigarettes are not expanding smoking.

In fact, the data in the paper suggest that if anything, the increase in vaping has been associated with an accelerated decline in smoking. The cut-off point of 2009 seems to have been selected to show no acceleration, but very few young people tried vaping in 2009. By 2011, only 1.5% of middle and high school students vaped within the past 30 days and the figures went up after that. If the decline in smoking over 2004-2011 is compared with the decline over following years, it may well have significantly accelerated.

The final conclusion that ‘E-cigarette–only users would be unlikely to have initiated tobacco product use with cigarettes’ makes no sense because e-cigarette only users have not initiated any tobacco product use!

If the authors mean by this that they initiated nicotine use, this is unlikely. In this as in other similar reports, smokers were asked on how many days they smoked in the past 30 days and it is most likely that the same question was asked of vapers, but these results are not reported. Studies that assessed frequency of use report that as with non-smokers who try nicotine replacement products such as nicotine chewing gum, it is extremely rare for non-smokers who try vaping to progress to regular use. While some smokers find e-cigarette satisfactory and switch to vaping, the majority of non-smokers who experiment with e-cigarettes only try them once or twice and virtually none progress to daily use.

On 2017 Feb 08, Lydia Maniatis commented:

To make clear what Adamian and Cavanagh (2017) do, and what they don’t do, in this publication. What they don’t do is to test a hypothesis. What they do is present a casual, ad hoc explanation of the Frohlich effect based on the results of past experiments, which they replicate here. The proposal remains untested. Even the ad hoc, untested assumptions (“we assume that the critical delay in producing the Fröhlich effect is not just the delay of attention in arriving at the target but also the time a saccade would then need to land on the target, if one were executed;”) can’t explain the results of their experiments, requiring more ad hoc proposals about complex processes: “The results suggest that the simultaneous onsets may be held in iconic memory and the cued motion trajectory can be retrieved if the cue arrives soon enough;” “A late SOA implies a longer memory retention period, and that means that the reported shifts could arise from working memory limitations and might not be perceptual in nature.”

Is Adamian and Cavanagh’s assumption that “the critical delay is not just the delay of attention….but also the time a saccade would then need to land on the target…” testable?

How would one go about testing it, as well as the additional assumptions the authors feel obliged to make with respect to memory?

Why didn’t the authors attempt to test their proposal to begin with, rather than simply performing replications that, even if successful, could do no more than leave the issue unresolved? They have not even proposed possible tests.

Obviously, replication was the safer choice, but one, again, that is essentially uninformative vis a vis an ad hoc proposal. It should be clear that the subject of eye movements and their role in perception is extremely complex and that casual speculations are unlikely to be borne out, if properly tested.

I think Adamian and Cavanagh’s proposal is so vague, the confounds so many, and (least of all, at present) the technical demands so great, that it cannot be tested. If all of the main and subsidiary assumptions, and their implications, were clarified enough to allow them to be critically assessed for logical coherence and consistency with other known facts, it might well fail at this stage, obviating the need for experimental tests.

Of course, I could be wrong in the present case; the authors may intend, post-replication, to attempt to concretize and subject their proposal to a genuine test; that would be genuinely refreshing.

I would note, as an afterthought, the uninformative nature of the title of the article, which is typical of many vision science articles and reflects the essentially uninformative nature of the work itself. The title tells us what the article is about, but not what it concluded or implied.

On 2017 Jan 24, Jim Johnson commented:

This paper is missing some highly relevant references from the Kieffer lab, including recent studies that establish the requirement for insulin in the anti-diabetic actions of leptin.

On 2017 Mar 08, Atanas G. Atanasov commented:

Very important and nicely summarized information that is of a very high relevance for the general public… I have featured this review at: http://healthandscienceportal.blogspot.com/2017/03/potential-benefits-and-harms-of-fasting.html

On 2017 Aug 08, Christopher Tench commented:

Can you possibly provide the coordinates used as it is not possible to understand exactly what analysis has been performed without them.

On 2017 Jan 31, Sin Hang Lee commented:

Assuming you have read the reference under the Comment.

Perhaps, you or someone would like to respond to the comment on behalf of Dr. Mark Schiffman and colleagues on the PubMed Commons below the abstract of following article. https://www.ncbi.nlm.nih.gov/pubmed/27905473

I would like to initiate a forum of open discussion, not one-sided proclamations.

On 2017 Jan 31, Stuart RAY commented:

The comment above does not provide any evidence for the dissent stated. What is "biased and dangerous" about the HPV vaccination recommendation?

On 2017 Jan 24, Sin Hang Lee commented:

The Editorial “Trump’s vaccine-commission idea is biased and dangerous” in Nature 2017 Jan 17;541(7637):259 is debatable. At least one article published by the Nature Publishing Group, in Nature Reviews Disease Primers 2016;2:16086 [1], promoting mass human papillomavirus (HPV) vaccination of girls 9-13 years of age and teenage boys at the cost of >$50 million for every 100,000 adolescents in the name of cervical cancer prevention is equally biased and dangerous. Medical journal censorship of dissenting data and opinions has suppressed the facts that the benefits of mass HPV vaccination are uncertain and the risks are substantial at great cost to society.

References:

1. https://www.ncbi.nlm.nih.gov/pubmed/27905473

Sin Hang Lee shlee01@snet.net Milford Molecular Diagnostics Laboratory Milford,CT

On 2017 Jan 23, Andy Collings commented:

(Original comment found at: https://elifesciences.org/content/6/e17044#disqus_thread)

Response to “Replication Study: Discovery and preclinical validation of drug indications using compendia of public gene expression data”

Atul J Butte, Marina Sirota, Joel T Dudley

We represent three of the key authors of the original work.

In October 2013, we were pleased to see that our original 2011 publication (Sirota et al., 2011) was selected as one of the top 50 influential cancer studies selected for reproducibility. Our initial impression, probably like most investigators reading this letter, was that such recognition would be a mixed blessing for us. Most of our work for this paper was conducted in 2009, 4 years prior to us being approached. We can see now that this reproducibility effort is one of the first 10 to be completed, and one of the first 5 to be published, more than 3 years later. The reproducibility team should be commended on their diligence to repeat experimental details as much as possible.

The goal of the original study was to evaluate a prediction from a novel systematic computational technique that used open-access gene-expression data to identify potential off-indication therapeutic effects of several hundred FDA approved drugs. We chose to evaluate cimetidine based on the biological novelty of its predicted connection to lung cancer and availability of local collaborators in this disease area.

The key experiment replicated here involved 18 mice treated with three varying doses of cimetidine (ranging from 25 to 100 mg/kg) administered via intraperitoneal injection daily to SCID mice after implantation of A549 human adenocarcinoma cells, along with 6 mice treated with doxorubicin as a positive control, and 6 mice treated only with vehicle as a negative control. The reproducibility team used many more mice in their experiment, but tested only the highest dose of cimetidine.

First, it is very important to clearly note that we are truly impressed with how much Figure 1 in the reproducibility paper matches Figure 4c in our original paper, and this is the key finding that cimetidine has a biological effect between PBS/saline (the negative control) and doxorubicin (the positive control). We commend the authors for even using the same colors as we did, to better highlight the match between their figure and ours.

While several valid analytic methods were used on the new tumor volume data, the analysis most similar to the original was the t-test we conducted on the measurements from day 11, with 100 mg/kg cimetidine compared to vehicle control. The new measurements were evaluated with a Welch t-test yielding t(53) = 2.16, with p=0.035. We are extremely pleased to see this raw p-value come out from their experiment.

However, the reproducibility team then decided to apply a Bonferroni adjustment, resulting in a corrected p=0.105. While this Bonferroni adjustment was decided a priori and documented (Kandela et al., 2015), we fundamentally do not agree with their approach.

The reproducibility team took on this validation effort by starting with our finding that cimetidine demonstrated some efficacy in the pre-clinical experiments. However, our study did not start with that prediction. We started our experiments with open data and a novel computational effort. Readers of our original paper (Sirota et al., 2011) will see that we started our study much earlier in the process, with publicly-available gene expression data on drugs and diseases, and computationally made predictions that certain drugs could be useful to treat certain conditions. We then chose cimetidine and lung adenocarcinoma from among the list of significant drug-disease pairs for validation. This drug-disease pairing was statistically significant in our computational analysis, which included the formal evaluation of multiple-hypothesis testing using random shuffled data and the calculation of q-values and false discovery rates. These are commonly used methods for controlling for the testing of multiple hypotheses. Aside from the statistical significance, local expertise in lung cancer and the availability of reagents and A549 cells and mouse models in our core facilities guided the selection. We then chose an additional pairing that we explicitly predicted (by the computational methodology) would fail. We again used cimetidine and found we had ACHN cells that could represent a model of renal cancer. Scientists will recognize this as a negative control.

At no point did we feel the comparison of cimetidine against A549 cells had anything to do with the effect of cimetidine in ACHN cells; these were independently run experiments. The ACHN cell test was to test the specificity of the computational process upstream of all of this; it had nothing to do with our belief in cimetidine in A549 cells. Thus, we would not agree with the replication team’s characterization that these were all multiple hypotheses being validated equally, and thus merited a common adjustment of p-values. As described above, we corrected for the multiple hypothesis testing earlier in our process, at the computational stage. We never expected the cimetidine/ACHN experiment to succeed when we ran it. Similarly, our test of doxorubicin in A549 cells was performed as a positive control experiment; we fully expected that experiment to succeed.

In email discussion, we learned the replication team feels these three hypotheses were tested equally, and thus adjusted the p-values by multiplying them by 3. We are going to have to respectfully “agree to disagree” here.

We note some interesting results of their adjustments, such as the reproducibility team also not finding doxorubicin to have a statistically significant effect compared to vehicle treated mice. Again, the Welch’s t-test on this comparison yielded p=0.0325, but with their Bonferroni correction, this would no longer be deemed a significant association. Doxorubicin has been used as a known drug against A549 cells for nearly 30 years (Nishimura et al, 1989), and our use of this drug was only as a positive-control agent.

Figure 3 was also very encouraging, where we do see a significant effect from the original and reproduced studies, and the meta-analysis together.

In the end, we want to applaud replication efforts like this. We do believe it is importance for the public to have trust in scientists, and belief in the veracity of our published findings. However, we do recommend replication teams of the future to choose papers in a more impactful manner. While it is an honor for our paper to be selected, we were never going to run a clinical trial of cimetidine in lung adenocarcinoma, and we cannot see any such protocol being listed in clinicaltrials.gov. Our publication was more towards demonstrating the value of open data, through the validation of a specific computational prediction. We suggest that future replication studies of pre-clinical findings should really be tailored towards those most likely to actually be heading into clinical trials.

References

Sirota M, Dudley JT, Kim J, Chiang AP, Morgan AA, Sweet-Cordero A, Sage J, Butte AJ. Discovery and preclinical validation of drug indications using compendia of public gene expression data. Sci Transl Med. 2011 Aug 17;3(96):96ra77. doi: 10.1126/scitranslmed.3001318.

Kandela I, Zervantonakis I; Reproducibility Project: Cancer Biology. Registered report: Discovery and preclinical validation of drug indications using compendia of public gene expression data. Elife. 2015 May 5;4:e06847. doi: 10.7554/eLife.06847.

Nishimura M, Nakada H, Kawamura I, Mizota T, Shimomura K, Nakahara K, Goto T, Yamaguchi I, Okuhara M. A new antitumor antibiotic, FR900840. III. Antitumor activity against experimental tumors. J Antibiot (Tokyo). 1989 Apr;42(4):553-7.

On 2017 Jan 20, Robert Tibshirani commented:

The Replication Study by Kandela et al of the Sirota et al paper “Discovery and Preclinical Validation of Drug Indications Using Compendia of Public Gene Expression Data“ reports a non-significant p-value of 0.105 for the test of the main finding for cimetidine in lung adenocarcinoma. They obtained this from a Bonferroni adjustment of the raw p-value of 0.035, multiplying this by three because the authors had also tested a negative and a positive control.

This seems to me to be an inappropriate use of a multiple comparison adjustment. These adjustments are designed to protect the analyst against errors in making false discoveries. However if Sirota et al had found that the negative control was significant, they would not have reported it as a "discovery". Instead, it would have pointed to a problem with the experiment. Similarly, the significant result in the positive control was not considered a "discovery" but rather was a check of the experiment's quality.

Now it is true that Kandela et al specified in their protocol that they would use a (conservative) Bonferroni adjustment in their analysis, and used this fact to choose a sample size of 28. This yielded an estimated power of 80%. If they had chosen to use the unadjusted test, the estimated power for n=28 would have been a little higher—about 90%. I think that the unadjusted test is appropriate here.

On 2017 Jan 23, Andy Collings commented:

(Original comment found at: https://elifesciences.org/content/6/e21634#disqus_thread)

Response to: “Replication Study: Melanoma genome sequencing reveals frequent PREX2 mutations"

Lynda Chin and Levi Garraway

We applaud the Reproducibility Project and support its goal to reproduce published scientific results. We also thank Horrigan et al for a carefully executed study, for which we provided reagents and extensive consultation throughout. Their work illustrates the inherent challenges in attempting to reproduce scientific results.

We summarize below the results of Horrigan et al., first in lay terms and then in more scientific detail.

Description for Lay Readers

Briefly, our 2012 paper reported that human melanoma patients often carry mutations in the PREX2 gene. To study the effect of mutations in PREX2, we made modified versions of a commonly used immortalized human melanocyte cell line (called p’mels) and injected them into mice. When mice were injected with cells carrying an irrelevant gene or a normal copy of PREX2 (“control mice”), tumors started to form in about 9 weeks. When mice were injected with cells carried the mutated PREX2 genes (“experimental mice”), tumors began to form after around 4-5 weeks—indicating that mutated PREX2 accelerated tumor formation.

When Horrigan et al. tried to reproduce our experiment, they found that tumors began to form in their control mice after about 1 week—not 9-10 weeks. Because their control developed tumors so rapidly, Horrigan et al. recognized that they could not meaningfully test our finding that mutant PREX2 accelerated the tumor formation.

Why did the human melanocyte cells grow tumors in the control mice so much faster in Horrigan et al.’s experiment? The likely explanation is that human cells engineered in this way are known to undergo dramatic changes when they are grown for extended periods in culture. Therefore, Horrigan et al.’s study underscores how important it is to have appropriate control cells, before attempting to reproduce experimental findings.

Finally, we emphasize that Horrigan et al.’s results do not call into question our results about PREX2 because their experiment was not informative. Moreover, we have recently validated the findings about PREX2 in an independent way—by creating genetically engineered mice that carry mutated PREX2 in their own genomes. These PREX2 mutant mice showed accelerated tumor growth compared to controls.

Description for Scientific Readers

The authors repeated a xenograft experiment (Figure 3b) in our 2012 report. In our experiment, we overexpressed GFP (negative control), wild type PREX2 (normal control) and two PREX2 mutants (G844D and Q1430*) (experimental arm) in a TERT-immortalized human melanocyte line engineered with RB and p53 inactivation (p’mel). To further sensitize these melanocytes for tumorigenicity, they were also engineered to overexpress oncogenic NRASG12D. We showed that the mutant PREX2 expression in p’mel cells significantly accelerated tumor formation in vivo. However, Horrigan et al found that the control and PREX WT or mutant expressing p’mels all behaved identically, forming tumors rapidly in vivo (within 1 week of implantation). This finding differed from our study, in which the control cells (both GFP and PREX2) did not form tumors until >10 weeks after implantation.

The fact that Horrigan et al observed rapid tumor formation in all settings means that their findings are uninformative with regard to the reproducibility of a central conclusion of our 2012 report, namely that mutant PREX2 can accelerate tumor formation in vivo. Testing this hypothesis requires a control arm in which tumor formation is sufficiently latent so that a discernible effect on the rate of tumorigenesis by the mutants can be observed. In the Horrigan et al study, tumorigenesis in the control arms was so rapid that it essentially became impossible to detect any additional effect of mutant PREX2.

Why were the controls so much more tumorigenic in the hands of Horrigan et al.? We note that although the investigators were provided with clones from the same p’mels used in the 2012 study, by the time Horrigan et al received the cells, more than two years had passed since the original p’mel cells were engineered. This is a crucial point, because as with many other cell lines, these “primed” human primary melanocytes are known to readily undergo adaptation during extended cultivation in vitro. In particular, these p’mels can spontaneously acquire a more transformed phenotype over time (we have seen this happen on multiple occasions). Thus, although a clone from the same engineered cells were provided to Horrigan et al, the fact that that clone of p’mel cells exhibited very different phenotype suggests that the additional passages, a major geographic relocation, and subsequent freeze-thaw manipulations have rendered them unsuitable as an experimental frame of reference.

When we notice such “drifts” in engineered cell culture models, we often have to re-derive the relevant lines starting from even earlier stages in order to have controls with suitable tumorigenic latency. For example, in this case, we would have re-introduced NRASG12D into a clone of non-transformed melanocytes harboring TERT immortalization and RB/P53 inactivation to re-engineer a p’mel cell line. Had Horrigan et al used less tumorigenic controls, they would have a much better chance to reproduce an accelerating effect of mutant PREX2.

To validate our initial observations regarding the oncogenic role of mutant PREX2, we have since taken an orthogonal approach: we created a genetically engineered mouse (GEM) model targeting both a truncating PREX2 mutation (E824*) and oncogenic NRASG12D expression to melanocytes under a tet-regulated promoter. In this GEM model, we observed significantly increased penetrance and decreased latency of melanoma formation (Lissanu Deribe et al PNAS, 2016, E1296-305; see Figure 3b in Lissanu Deribe et al PNAS, 2016), thus confirming the xenograft findings of our 2012 report showing that mutant PREX2 is oncogenic.

In summary, we support rigorous assessments of reproducibility such as this. Equally, we consider it crucial to recognize and account for salient underlying properties of the model systems and experimental controls in order to minimize the risk of misleading conclusions regarding the reproducibility of any given experiment. Indeed, Horrigan et al. nicely articulated the importance of these considerations when discussing their results.

On 2017 Jan 23, Andy Collings commented:

(Original comment in full found at: https://elifesciences.org/content/6/e18173#disqus_thread)

Response to: “Replication Study: The CD47-signal regulatory protein alpha (SIRPa) interaction is a therapeutic target for human solid tumors”

Irving Weissman for the authors of "The CD47-signal regulatory protein alpha (SIRPa) interaction is a therapeutic target for human solid tumors"

Our original paper by Willingham and Volkmer et al in PNAS reported the result of experiments testing the hypothesis that CD47 might be expressed and demonstrate dominant ‘don’t eat me’ functions on human solid cancers, as well as our previously described studies with human leukemias and lymphomas and mouse leukemias. The study included primarily experiments on primary patient solid cancers with minimal passage as xenografts in immune deficient mice tested in vitro and as xenografts in the mice lacking adaptive immune system T, B, and NK cells, but possessing all other bone marrow derived innate immune system cells such as macrophages . We included one experiment on a long passaged mouse breast cancer line transplanted into syngeneic immunocompetent FVB mice. The Replication Study by Horrigan et al in eLife reports the results of efforts to repeat the experiments on the passaged mouse breast cancer line, but none of the experiments on human primary and minimally passaged cancers of several different solid tumor types, either in vitro or as xenografts in mice in which the CD47 ‘don’t eat me’ signal was blocked with monoclonal antibodies.

When we were requested to participate in a replication study of our paper entitled “The CD47-signal regulatory protein alpha (SIRPa) interaction is a therapeutic target for human solid tumors” we agreed, but were worried that we had spent years developing the infrastructure to obtain human cancers from de-identified patients, found ways to transplant them into immune deficient mice, and limited our studies to human cancers within 1 to less than 10 transplant passages in these mice. Our major objective in the study was to test whether the CD47 molecule was present on these human solid tumors, if it acted as a ‘don’t eat me’ signal for mouse and human macrophages, and whether these tumors in immune deficient mice were susceptible to blocking anti-CD47 antibodies. This was a scientific paper to answer these questions, and not a preclinical study preparatory to human clinical trials.

To our surprise, our study verified on all tested human cancers that they express CD47, perhaps the first cancer gene commonly expressed on all cancers; and it is a molecule which provides a ‘don’t eat me’ function; and we showed that blocking that function led to tumor attack by macrophages.

Unfortunately, the independent group who accepted the task of replicating our studies did not do a single study with human cancers, or to study the effect of our blocking antibodies to the CD47 tumor cell surface molecules on the phagocytic removal of human cancers.

Horrigan et al did begin, with our help, to replicate the one study we did as a pilot to see if anti-CD47 antibodies that also bind to mouse CD47 would have an effect on a long-transplanted mouse breast cancer line. We and others have found that the exact way you transplant these mouse cancers is critical to achieve engraftment of the cancers in appropriate immune competent mice. As we learned from Dr Sean Morrison, UT Southwestern Childrens Hospital, many cancers won’t grow in mice unless a special type of matrigel is used to support the cells in vitro and in transplant. Without it, transplantation may be sporadic and/or absent. The replication team found their own matrigel, and for reasons unknown to us, could not get reproducible transplantation in their testing. This was picked up in reviews of the paper by eLife referees, including a request for repeating the studies a number of ways, but that did not happen.

There is therefore no study that addresses the title of the paper and its major conclusions: human cancers express CD47 and our studies show that it is a target for therapeutic studies.

Several independent papers since ours have replicated not only our findings, but have extended them to many other human cancers (see below). So replication of our major points have occurred with independent groups.

But we agree that everything we publish, major or minor, central or peripheral, must be replicable. Even in our human tumor studies there were a few outlier cancers that did not diminish growth in the presence of blocking anti-CD47 antibodies.

The beginning of replication is to show experience and competence in the transplantability of the cancer. There are many possible reasons that replication of the basic transplantation of MT1A2 breast cancer cells in syngeneic FVB mice was not replicated in the experiments carried out by Horrigan et al, who got only a fraction of the mice transplanted. These could include the particular matrigel used, a problem with using long passaged cell lines[which may be heterogeneous and altered by the passaging in vitro and in vivo], rather than primary or recent mouse or human cancers. It could be inherent in how Horrigan et al did the experiments. Oddly, the control antibodies did diminish the growth of the MT1A2 cancers in their single experiment. Amongst the reasons concerning the heterogeneity of long passaged cell lines we might cite is that we have discovered two more ‘don’t et me’ molecules on cancers that interact with other receptors on macrophages. Although those papers are submitted, but not yet published, we cannot specify the details lest we endanger their publishability. (Readers who send us a request will receive copies of the papers when published.) Laboratories that study tumors at different transplant passages have often found that variant subsets of cells within the cancer can rapidly outgrow the major population of cells transplanted, and it is common that the successive transplants grow more aggressively in the same strain of mice, even though the name of the tumor is retained. For that reason it is clear that studies on long-passaged tumors may be studying some properties of the passaged cell rather than the original cancer in the individual. There are other possibilities. When the replication study lab interacted with us early on, we offered to do the experiments side by side with them to facilitate technology transfer. Horrigan et al declined. The offer still stands.

Before this paper was published we published other papers demonstrating that CD47 was expressed on all samples of human AML and human NHL tested, usually at a higher level than on the same stage or type of human normal cell. Further, we showed both by genetic manipulation of the expression of CD47 on human cells or the treatment of those cells with blocking antibodies to CD47 that interrupt its interaction with macrophage receptor Sirpα lead to mouse or human macrophages to phagocytose and kill the tumor target cells. We used anti-human antibodies that did not trigger phagocytosis by ‘opsonization’, as the isotype of the antibodies used for blocking were not the isotype that is highly efficient at triggering complement activation or ADCC (activation via Fc receptor of NK lineage killer cells), and we demonstrated that on human lymphomas. [...]

(The comment in full can be found at: https://elifesciences.org/content/6/e18173#disqus_thread)

On 2017 Aug 29, Laura M Cox commented:

Thank you for catching this typo. The journal will issue a corrigendum soon.

On 2017 Aug 24, Seán Turner commented:

In the title, "Faecalibacterium (sic) rodentium" should be "Faecalibaculum rodentium."

On 2017 Feb 22, Tom Yates commented:

Shah and colleagues are to be congratulated for an important study (Shah NS, 2017), emphasising the major role of transmitted resistance in the epidemiology of extensively drug-resistant tuberculosis (XDR-TB). However, methodological issues will have impacted the results.

As the authors acknowledge, in such studies, missing data bias estimates, with linked isolates wrongly designated unique. Their decision to look at a convenience sample of 51% of cases from throughout KwaZulu-Natal rather than attempt complete enrolment in a smaller area will have accentuated this bias.

A growing body of research suggests transmission between members of the same household only explains a small proportion of all Mycobacterium tuberculosis (MTB) transmission in Sub Saharan Africa (Verver S, 2004, Andrews JR, 2014, Middelkoop K, 2015, Glynn JR, 2015). In the present study, recall bias plus disease prompting contacts to test for XDR-TB will likely have resulted in household, workplace and hospital contacts being captured more consistently than more casual community contacts.

Determining the proportion of total MTB transmission occurring in specific locations would allow disease control programmes to be better targeted. I agree with Shah et al that this should be a research priority.

Dr Tom A. Yates, Institute for Global Health, University College London, t.yates@ucl.ac.uk

On 2017 Jan 24, Pushkar Malakar commented:

This study has the potential to open a new field to explore like viral communications or signaling in viruses.Someway down the line there is possibility that one will know about the evolution of communication system or the signaling system between two organisms.This study has therapeutic potential also as viruses are responsible for many fatal human diseases like cancer, AIDS etc.If one understand the communication or signaling system between viruses then therapeutics can be developed to block this communication or signaling system which will help in preventing the viral diseases.More understanding of viral communication or signaling system may be used in biotech industries to produce cheaper and useful bioproducts as viruses replicate very fast. Further viruses might use different signaling molecules to communicate with each other for different activities.Signaling system might also help in the classification of viruses. Anyway this study is just the beginning and many more mysteries about viruses might be solved in the near future.

On 2017 Feb 06, Andrea Giaccari commented:

PubMed states this article is Free Full Text, but then links to http://www.bmj.com/content/356/bmj.i6505 asking for "Article access for 1 day: Purchase this article for £23 $37 €30 + VAT"

On 2017 Aug 05, Jon-Patrick Allem commented:

There are at least five problems with this paper: First, the authors simply assume that the pro-e-cigarette tweets are wrong and need their corrective input. What if users are right to be positive? The authors have not demonstrated any material risk from vapour aerosol. To the extent that there is evidence of exposure the levels so low as to be very unlikely to be a health concern. The presence of a hazardous agent does not in itself imply a risk to health, there has to be sufficient exposure to be toxicologically relevant.

This critique is misguided. The goal of this paper was to characterize public perception of e-cigarette aerosol by using a novel data source (tweets) and not to demonstrate any material risk from e-cigarette aerosol.

Second, they have also not considered what harmful effect that their potentially misleading 'health education messages' may have. For example, by exaggerating a negligible risk they may be discouraging people from e-cigarette use, and potentially causing relapse to smoking and reducing the incentive to switch - thus doing more harm than had they not intervened. We already know the vast majority of smokers think e-cigarettes are much more dangerous than the toxicological profile of the aerosol suggests - see National Cancer Institute HINTS data. The authors' ideas would aggravate these already highly damaging misperceptions of risk.

This critique is misguided. This study did not design educational messages. It described people’s perceptions about e-cigarette aerosol.

Third, as so often happens with tobacco control research, the authors make a policy proposal for which their paper comes nowhere close to providing an adequate justification. Public health and regulatory agencies could use social media and traditional media to disseminate the message that e-cigarette aerosol contains potentially harmful chemicals and could be perceived as offensive. They have not even studied the effects of the messages they are recommending on the target audience or tested such messages through social media. If they did, they would discover that users are not passive or compliant recipients of health messages, especially if they suspect they are wrong or ill-intentioned. Social media creates two-way conversations in which often very well-informed users will respond persuasively to what they find to be poorly informed or judgemental health messages. Until the authors have tested a campaign of the type they have in mind, they have no basis for recommending that agencies spend public money in this way.

This critique is misguided. There was no policy proposal made in the passage highlighted here. The suggestion that social media platforms can be used as a communication channel is not a policy. It is a communication strategy. The idea that social media can be used to obtain information and later communicate messages is completely in line with the work presented in this paper. The notion that every paper answers every research question pertaining to a topic is an unreasonable expectation.

Fourth, the authors suggest that users should be warned by public health agencies that "e-cigarette aerosol ... could be perceived as offensive". If there were warnings from public health and regulatory agencies about everything that could be perceived as offensive by someone, then we would be inundated with warnings. This is not a reliable basis or priority for public health messaging. Given the absence of any demonstrable material risk from e-cigarette aerosol, the issue is one of etiquette and nuisance. This does not require government intervention of any sort. Vaping policy in any public or private place should be a matter for the owners or managers, who may not find it offensive nor wish to offend their clientele. It is not a matter for legislators, regulators or health agencies.

This critique here is based on one’s own opinion about the role of government and could be debated with no clear stopping point.

Fifth (and with thanks to Will Moy's tweet), the work is pointless and wasteful. Who cares what people are saying on twitter about e-cigarettes and secondhand aerosol exposure? Why is this even a subject worthy of study and what difference could it make to any outcomes that are important for health or any other policy? What is the rationale for spending research funds on this form of vaguely creepy social media surveillance? Updated 21-Jan-17 with fifth point.

Big social media data (Twitter, Instagram, Google Webs Search) can be used to fill certain knowledge gaps quickly. While one study using one data source is by no means definitive, one study based on timely data can provide an important starting-off point to address an issue of great import to public health. This paper describes why understanding public sentiment toward e-cigarette aerosol is relevant and utilizes a data source that allowed people to organically report on their sentiment toward e-cigarette aerosol unprimed by a researcher, without instrument bias, and at low costs. Also, policy development and communication campaigns are two distinct areas of research. The goal of this study was to inform the latter.

On 2017 Jan 25, Erica Melief commented:

None

On 2017 Jan 21, Clive Bates commented:

There are at least five problems with this paper:

First, the authors simply assume that the pro-e-cigarette tweets are wrong and need their corrective input. What if users are right to be positive? The authors have not demonstrated any material risk from vapour aerosol. To the extent that there is evidence of exposure the levels so low as to be very unlikely to be a health concern. The presence of a hazardous agent does not in itself imply a risk to health, there has to be sufficient exposure to be toxicologically relevant.

Second, they have also not considered what harmful effect that their potentially misleading 'health education messages' may have. For example, by exaggerating a negligible risk they may be discouraging people from e-cigarette use, and potentially causing relapse to smoking and reducing the incentive to switch - thus doing more harm than had they not intervened. We already know the vast majority of smokers think e-cigarettes are much more dangerous than the toxicological profile of the aerosol suggests - see National Cancer Institute HINTS data. The authors' ideas would aggravate these already highly damaging misperceptions of risk.

Third, as so often happens with tobacco control research, the authors make a policy proposal for which their paper comes nowhere close to providing an adequate justification.

Public health and regulatory agencies could use social media and traditional media to disseminate the message that e-cigarette aerosol contains potentially harmful chemicals and could be perceived as offensive.

They have not even studied the effects of the messages they are recommending on the target audience or tested such messages through social media. If they did, they would discover that users are not passive or compliant recipients of health messages, especially if they suspect they are wrong or ill-intentioned. Social media creates two-way conversations in which often very well-informed users will respond persuasively to what they find to be poorly informed or judgemental health messages. Until the authors have tested a campaign of the type they have in mind, they have no basis for recommending that agencies spend public money in this way.

Fourth, the authors suggest that users should be warned by public health agencies that "e-cigarette aerosol ... could be perceived as offensive". If there were warnings from public health and regulatory agencies about everything that could be perceived as offensive by someone, then we would be inundated with warnings. This is not a reliable basis or priority for public health messaging. Given the absence of any demonstrable material risk from e-cigarette aerosol, the issue is one of etiquette and nuisance. This does not require government intervention of any sort. Vaping policy in any public or private place should be a matter for the owners or managers, who may not find it offensive nor wish to offend their clientele. It is not a matter for legislators, regulators or health agencies.

Fifth (and with thanks to Will Moy's tweet), the work is pointless and wasteful. Who cares what people are saying on twitter about e-cigarettes and secondhand aerosol exposure? Why is this even a subject worthy of study and what difference could it make to any outcomes that are important for health or any other policy? What is the rationale for spending research funds on this form of vaguely creepy social media surveillance?

Updated 21-Jan-17 with fifth point.

On 2017 Jan 21, Clive Bates commented:

How did the author manage to publish a paper with the title "E-cigarettes: Are they as safe as the public thinks?", without citing any data on what the public actually does think? There is data in the National Cancer Institute's HINTS survey 2015. This is what it says:

Compared to smoking cigarettes, would you say that electronic cigarettes are…

• 5.3% say much less harmful
• 20.6% say less harmful
• 32.8% say just as harmful
• 2.7% say more harmful
• 2.0% say much more harmful
• 1.2% have never heard of e-cigarettes
• 33.9% don’t know enough about these products

Which brings me to the main issue with the paper. The author claims that there is insufficient knowledge to determine if these products are safer than cigarettes. This is an extraordinary and dangerous claim given what is known about e-cigarettes and cigarettes. It is known with certainty that there are no products of combustion of organic material (i.e tobacco leaf) in e-cigarette vapour - this is a function of the physical and chemical processes involved. We also know that products of combustion cause almost all of the harm associated with smoking. There is also extensive measurement of harmful and potentially harmful constituent of cigarette smoke and e-cigarette aerosol showing many are not detectable or present at levels two orders of magnitude lower in the vapour aerosol (e.g. see Farsalinos KE, 2014, Burstyn I, 2014). So the emissions are dramatically less toxic and exposures much lower.

The author provides a familiar non-sequitur: "There are no current studies that prove that e-cigarettes are safe". There never will be. Firstly because it is impossible to prove something to be completely safe, and almost nothing is. Secondly, no serious commentators claim they are completely safe, just very much safer than smoking. Hence the term 'harm reduction' to describe the benefits of switching to these products.

This view commands support in the expert medical profession. The Royal College of Physicians (London) assessed the toxicology evidence in its 2016 report Nicotine without smoke: tobacco harm reduction and concluded:

Although it is not possible to precisely quantify the long-term health risks associated with e-cigarettes, the available data suggest that they are unlikely to exceed 5% of those associated with smoked tobacco products, and may well be substantially lower than this figure. (Section 5.5 page 87)

This is a carefully measured statement that aims to provide useful information to both users of the products and health and medical professionals while reflecting residual uncertainty. It contrasts with the author's information leaflet for patients, which even suggests there is no basis for believing e-cigarettes to be safer than smoking:

If you are smoking and not planning to quit, we don't know if e-cigarettes are safer. Talk to your health care provider.

But we do know beyond any reasonable doubt that e-cigarettes are very much safer - the debate is whether they are 90% safer or 99.9% safer than smoking. Regrettably, only 5.3% of American adults correctly believe that e-cigarettes are very much less harmful than smoking, while 37% incorrectly think they are as harmful or more harmful (see above). The danger with these misperceptions of risk is that they affect behaviour, causing people to continue to smoke when they might otherwise switch to much safer vaping. The danger with a paper like this and its patient-facing leaflet is that it nurtures these harmful risk misperceptions and becomes, therefore, a vector for harm.

To return to the author's title question: E-Cigarettes: Are They as Safe as the Public Thinks?. The answer is: "No, they are very much safer than the public thinks".

On 2017 Jan 22, Eric Fauman commented:

I know nothing about cow genetics, but I have done some work on the genetics of metabolites in humans, so I was interested to see how the authors derived biological insights from this genetic study. In particular, I was intrigued by the suggestion in the abstract that they found evidence that genes involved in the synthesis of “milk components” are important for lactation persistence.

Unfortunately, the more I studied the paper the more problems I found that call this claim into question.

First off, the Q-Q plot is currently unavailable, but the text mentions there’s only a “slight deviation in the upper right tail”, which could mean there are no true significant signals.

To account for multiple testing, the authors decided to use a genome-wide association p-value cutoff of 0.95/44100 = 2.15e-5 instead of a more defensible 0.05/44100 = 1.1e-6.

Since their initial p-value cutoff yielded a relatively small number of significant SNPs, the authors used a much more lenient p-value cutoff of 5e-4 which presumably is well within the linear portion of the Q-Q plot.

The biggest problem with the enrichment analysis, however, is that they’ve neglected to account for genes drawn from a common locus. Often, paralogs of similar function are proximal in the genome. But typically we assume that a single SNP is affecting the function of only a single gene at a locus. So, for example, a SNP near the APOA4/APOA1/APOC3/APOA5 locus can tag all 4 genes, but it’s unfair to consider that 4 independent indications that “phospholipid efflux”, “reverse cholesterol transport”, “triglyceride homeostasis” and other pathways are “enriched” in this GWAS.

This issue, of overcounting pathways due to gene duplication, affects all their top findings, presumably rendering them non-significant. Besides lipid pathways, this issue also pertains to the “lactation” GO term, which was selected based on the genes GC, HK2, CSN2 and CSN3. GC, CSN2 and CSN3 are all co-located on Chromosome 6.

A perplexing claim in the paper is for the enrichment of the term “lipid metabolic process” (GO:0006629). According to the Ensembl Biomart, 912 Bos taurus genes fall into this category, or about 4% of the bovine protein coding genes (24616 according to Ensembl). So out of their set of 536 genes (flanking SNPs with P < 5e-4) we’d expect about 20 “lipid metabolic process” genes. And yet, this paper reports only 7. This might be significant, but for depletion, not enrichment.

Sample size is of course a huge issue in GWAS. While 3,800 cows is a large number, it appears this trait may require a substantially larger number of animals before it can yield biologically meaningful results.

On 2017 Jan 16, Stuart RAY commented:

This is a scientifically interesting report, but the use of "mutation rate" in the title, abstract, and in some portions of the text is unfortunate, because the process being observed and measured in this report is evolutionary rate of substitution (as noted in the authors' Tables 1 and 2). The evolutionary rate of substitution results from a variety of processes that are affected by the rate mutation (determined in particular by the polymerase), positive and negative selection, and stochastic events at multiple levels (from individual cell to population). Thus, the term "mutation rate" is confusing and potentially misleading. With every RNA genome replication, there is a nonzero rate of mutation; what we estimate when we sequence virus obtained from infected individuals, sampled over a period of years, is the evolutionary rate of substitution.

On 2017 Jun 24, Shafic Sraj commented:

Cubital tunnel score in the presence of carpal tunnel syndrome

On 2017 Jan 21, Thomas Heston commented:

This appears to be a classic example of the Hawthorne Effect, i.e. what gets examined tends to improve (http://www.economist.com/node/12510632). The conclusion of this research seems to be that focusing on a problem by providing feedback tends to improve that problem, compared to doing nothing.

On 2017 Jan 31, Stuart RAY commented:

This is a very interesting and well-done study of a model system. That said, the unqualified use of the terms "antiviral effects of IFN-lambda" and "norovirus" in the title of this article might be misleading without context. Readers should be alert: (a) the noroviruses are very diverse biologically and phylogenetically; (b) murine norovirus is distinct from human noroviruses in apparent tropism, binding (sialic vs blood group antigens), pH dependence of viral entry (Kim Y. Green, Fields Virology 2013, chapter 20); (c) there are significant biological differences between human and mouse responses to lambda interferon (Hermant P, 2014); and B6 mice lack functional MX1 (Pillai PS, 2016, Moritoh K, 2009). Given differences in virus and host, whether the findings presented by Baldridge et al. can be extrapolated to other systems (e.g. natural human norovirus infection) is highly uncertain; therefore, I suggest that the title should end with "in mice".

On 2017 Jan 14, Sin Hang Lee commented:

In a research paper titled “Specific microbiota direct the differentiation of IL-17-producing T-helper cells in the mucosa of the small intestine” published by Ivanov et al. in Cell Host Microbe. 2008 Oct 16;4(4):337-49, antibiotic treatment of the specific microbiota has been shown to inhibit TH17 cell differentiation. Perhaps, Strle and colleagues may consider developing microbiological tests for accurate diagnosis of the early infection of Lyme borreliosis in patients with or without skin lesions for timely appropriate antibiotic treatment to prevent excessive TH17 responses and the subsequent autoimmune disorders.

On 2017 Feb 05, Kevin Hall commented:

The theoretical basis of the carbohydrate-insulin model (CIM) relies on generally accepted physiology about endocrine regulation of adipose tissue– data that were all collected on short time scales. Ludwig appears to suggest that this long debate has been about a “straw man” short-term version of the CIM. This apparently explains why the purported metabolic advantages have been elusive when assessed by inpatient controlled feeding studies that were simply too short to unveil the metabolic advantages of the CIM. Indeed, Ludwig believes he has scored a win in this debate by acknowledging that these metabolic advantages of low carbohydrate diets on energy expenditure and body fat predicted by the CIM must operate on longer time scales, conveniently where no inpatient data have been generated either supporting or negating those predictions. This was accurately described in my review as an ad hoc modification of the CIM – a possibility currently unsupported by data but obviously supported by sincere belief.

On 2017 Feb 05, DAVID LUDWIG commented:

With Hall’s comment of 4 Feb 2017, this long debate nears resolution. He acknowledges it’s “possible” that the very short metabolic studies do not reflect the long-term effects of macronutrients on body weight. We disagree on how likely that possibility is, and now must await further research to resolve the scientific uncertainties.

Finally, on an issue of academic interest only, Hall creates a straw man in claiming to have “falsified” the Carbohydrate-Insulin Model (CIM). Versions of CIM were originally proposed more than a century ago, as detailed by Taubes G, 2013, before short term studies of substrate oxidation would have been possible. Furthermore, in the second paragraph of his review, Hall cites an article I coauthored Ludwig DS, 2014 and three by others Lustig RH, 2006, Taubes G, 2013, Wells JC, 2011 as recent iterations of CIM. Each of these articles focuses on long-term effects, and none asserts that 1 week should be adequate to prove or falsify CIM. In view of the failure of conventional approaches to address the massive public health challenge of obesity, let’s now refocus our energies into the design and execution of more definitive research.

On 2017 Feb 05, Kevin Hall commented:

Ludwig suggests that demonstration of any metabolic adaptations occurring on a time scale of > 1 week after introduction of an isocaloric low carbohydrate diet somehow invalidates all of the inpatient controlled feeding studies with results that violate carbohydrate-insulin model (CIM) predictions. This presents a false dilemma and is a red herring.

There are indeed metabolic adaptations that take place on longer time scales, but many of these changes actually support the conclusion that the purported metabolic advantages for body fat loss predicted by the CIM are inconsistent with the data. For example, as evidence for a prolonged period of fat adaptation, Ludwig notes modest additional increases in blood and urine ketones observed after 1 week of either starvation Owen OE, 1983 or consuming a hypocaloric ketogenic diet Yang MU, 1976. The implication is that daily fat and ketone oxidation presumably increase along with their blood concentrations over extended time periods to eventually result in an acceleration of body fat loss with low carbohydrate high fat diets as predicted by the CIM. But since acceleration of fat loss during prolonged starvation would be counterproductive to survival, might there be data supporting a more physiological interpretation the prolonged increase in blood and urine ketones?

Both adipose lipolysis Bortz WM, 1972 and hepatic ketone production Balasse EO, 1989 reach a maximum within 1 week as demonstrated by isotopic tracer data. Therefore, rising blood ketone concentrations after 1 week must be explained by a reduced rate of removal from the blood. Indeed, muscle ketone oxidation decreases after 1 week of starvation and, along with decreased overall energy expenditure, the reduction in ketone oxidation results in rising blood concentrations and increased urinary excretion (page 144-152 of Burstztein S, et al. ‘Energy Metabolism, Indirect Calorimetry, and Nutrition.’ Williams & Wilkins 1989). Therefore, rather than being indicative of progressive mobilization of body fat to increase oxidation and accelerate fat loss, rising concentrations of blood ketones and fatty acids occurring after 1 week arise from reductions in ketone and fat oxidation concomitant with decreased energy expenditure.

The deleterious effects of a 600 kcal/d low carbohydrate ketogenic diet on body protein and lean mass were demonstrated in Vasquez JA, 1992 and were found to last about 1 month. Since weight loss was not significantly different compared to an isocaloric higher carbohydrate diet, body fat loss was likely attenuated during the ketogenic diet and therefore in direct opposition to the CIM predictions. Subsequent normalization of nitrogen balance would tend to result in an equivalent rate of body fat loss between the isocaloric diets over longer time periods. In Hall KD, 2016, urinary nitrogen excretion increased for 11 days after introducing a 2700 kcal/d ketogenic diet and coincided with attenuated body fat loss measured during the first 2 weeks of the diet. The rate of body fat loss appeared to normalize in the final 2 weeks, but did not exceed the fat loss observed during the isocaloric high carbohydrate run-in diet. Mere normalization of body fat and lean tissue loss over long time periods cannot compensate for early deficiencies. Therefore, these data run against CIM predictions of augmented fat loss with lower carbohydrate diets.

While I believe that outpatient weight loss trials demonstrate that low carbohydrate diets often outperform low fat diets over the short-term, there are little body weight differences over the long-term Freedhoff Y, 2016. However, outpatient studies cannot ensure or adequately measure diet adherence and therefore it is unclear whether greater short-term weight losses with low carbohydrate diets were due to reduced diet calories or the purported “metabolic advantages” of increased energy expenditure and augmented fat loss predicted by the CIM. The inpatient controlled feeding studies demonstrate that the observed short-term energy expenditure and body fat changes often violate CIM predictions.

Ludwig conveniently suggests that all existing inpatient controlled feeding studies have been too short and that longer duration studies might produce results more favorable to the CIM. But even this were true, the current data demonstrating repeated violations of CIM model predictions constitute experimental falsifications of the CIM requiring an ad hoc modification of the model such that the metabolic advantages only begin after a time lag lasting many weeks. This is possible, but unlikely.

On 2017 Feb 04, DAVID LUDWIG commented:

Boiling down his comment of 3 Feb 2017, Hall disputes that the metabolic process of adapting to a high-fat/low-carbohydrate diet confounds interpretation of his and other short term feeding studies. If we can provide evidence that this process could take ≥ 1 week, the last leg of his attack on the Carbohydrate-Insulin Model collapses. Well, a picture is worth a thousand words, and here are 4:

For convenience, these figures can be viewed at this link:

Owen OE, 1983 Figure 1. Ketones are, of course, the hallmark of adaptation to a low-carbohydrate ketogenic diet. Generally speaking, the most potent stimulus of ketosis is fasting, since the consumption of all gluconeogenic precursors (carbohydrate and protein) is zero. As this figure shows, the blood levels of each of the three ketone species (BOHB, AcAc and acetone) continues to rise for ≥3 weeks. Indeed, the prolonged nature of adaptation to complete fasting has been known since the classic starvation studies of Cahill GF Jr, 1971. It stands to reason that this process might take even longer on standard low-carbohydrate diets, which inevitably provide ≥ 20 g carbohydrate/d and substantial protein.

Yang MU, 1976 Figure 3A. Among men with obesity on an 800 kcal/d ketogenic diet (10 g/d carbohydrate, 50 g/d protein), urinary ketones continued to rise for 10 days through the end of the experiment, and by that point had achieved levels equivalent only to those on day 4 of complete fasting. Presumably, this process would be even slower with a non-calorie restricted ketogenic diet (because of inevitably higher carbohydrate and protein content).

Vazquez JA, 1992 Figure 5B. On a conventional high-carbohydrate diet, the brain is critically dependent on glucose. With acute restriction of dietary carbohydrate (by fasting or a ketogenic diet), the body obtains gluconeogenic precursors by breaking down muscle. However, with rising ketone concentrations, the brain becomes adapted, sparing glucose. In this way, the body shifts away from protein to fat metabolism, sparing lean tissue. This phenomenon is clearly depicted among women with obesity given a calorie-restricted ketogenic diet (10 g carbohydrate/d) vs a nonketogenic diet (76 g carbohydrate/d), both with protein 50 g protein/d. For 3 weeks, nitrogen balance was strongly negative on the ketogenic diet compared to the non-ketogenic diet, but this difference was completely abolished by week 4. What would subsequently happen? We simply can’t know from the short-term studies.

Hall KD, 2016 Figure 2B. Hall’s own study shows that the transient decrease in rate of fat loss upon initiation of the ketogenic diet accelerates after 2 weeks.

The existence of this prolonged adaptive process explains why metabolic advantages for low-fat diet are consistently seen in very short metabolic studies. But after 2 to 4 weeks, advantages for low-carbohydrate diets begin to emerge, as summarized in my comment of 3 Feb 2017, below.

Fat adaptation on low-carbohydrate diets has admittedly not been thoroughly studied, and its duration may differ among individuals and between experimental conditions. Nevertheless, there is strong reason to think that short feeding studies (i.e., < 3 to 4 weeks) have no relevance to the long-term effects of macronutrients on metabolism and body composition.

On 2017 Feb 04, Kevin Hall commented:

Is it really an “extreme argument” to conclude that important aspects of the carbohydrate-insulin model (CIM) have been falsified based on data from 20 highly controlled inpatient human feeding studies that failed to support key CIM model predictions? While previously ignoring this conforming body of data from other research groups, Ludwig now conveniently concludes that all of these studies were flawed in some way and are therefore irrelevant and incapable of testing any aspect of the carbohydrate-insulin model.

To Ludwig, more relevant for assessing the energy expenditure and body fat predictions of the CIM are rodent studies and outpatient human studies where diet adherence cannot be adequately controlled or assessed Winkler JT, 2005. One such study Ludwig uses to bolster the CIM Ebbeling CB, 2012 did not measure body fat during the test diets and showed no significant energy expenditure differences between diets with the same amount of protein but varying in carbohydrate vs. fat. Ludwig claims that the study supports the CIM because energy expenditure was observed to increase with a very low carbohydrate diet. But the concomitant 50% increase in protein vs. the comparator diets makes it impossible to definitively conclude that any observed effect was due to carbohydrate reduction alone. Ludwig’s arguments about the possibly minimal effects of dietary protein changes on energy expenditure cannot eliminate this important confound.

Ludwig argues that “adaptation to a higher-fat diet can take at least a week and perhaps considerably longer”. One of Ludwig’s citations in this regard describes the role of diet composition on fuel utilization and exercise performance Hawley JA, 2011. This review paper reported that adaptation to a high fat diet for <1 week was sufficient to alter fuel utilization, but 4-7 days of fat adaptation was required to maintain subsequent exercise performance. Interestingly, longer periods of fat adaptation during training (7 weeks) were concluded to limit exercise capacity and impair exercise performance. The other two studies Ludwig cited to support the necessity for long term fat adaptation fail to support the CIM. An inpatient controlled feeding study Vasquez JA, 1992 showed that a very low carbohydrate, high fat diet led to significantly greater loss of body protein and lean tissue mass despite no significant difference in weight loss compared to an isocaloric higher carbohydrate, lower fat diet. The second study was an outpatient feeding trial Velum VL, 2017 that failed to demonstrate a significant difference in body weight or fat loss despite prescribing diets substantially varying in carbohydrate vs. fat for 3 months.

I agree with Ludwig that it likely takes a long time to equilibrate to added dietary fat without simultaneously reducing carbohydrate because, unlike carbohydrate and protein, dietary fat does not directly promote its own oxidation and does not significantly increase daily energy expenditure Schutz Y, 1989 and Horton TJ, 1995. Unfortunately, these observations also run counter to CIM predictions because they imply that added dietary fat results in a particularly efficient means to accumulate body fat compared to added carbohydrate or protein Bray GA, 2012. If such an added fat diet is sustained, adipose tissue will continue to expand until lipolysis is increased to sufficiently elevate circulating fatty acids and thereby increase daily fat oxidation to reestablish balance with fat intake Flatt JP, 1988.

In contrast, when added fat is accompanied by an isocaloric reduction in carbohydrate, daily fat oxidation plateaus within the first week as indicated by the rapid and sustained drop in daily respiratory quotient in Hall KD, 2016 and Schrauwen P, 1997. Similarly, Hall KD, 2015 observed a decrease and plateau in daily respiratory quotient with the reduced carbohydrate diet, whereas the reduced fat diet resulted in no significant changes indicating that daily fat oxidation was unaffected. As further evidence that adaptations to carbohydrate restriction occur relatively quickly, adipose tissue lipolysis is known to reach a maximum within the first week of a prolonged fast Bortz WM, 1972 as does hepatic ketone production Balasse EO, 1989.

While there is no evidence that carbohydrate restricted diets lead to an acceleration of daily fat oxidation on time scales longer than 1 week, and there is no known physiological mechanism for such an effect, this possibility cannot be ruled out. Such speculative long term effects constitute an ad hoc modification of the carbohydrate-insulin model whereby repeated violations of model predictions on time scales of 1 month or less are somehow reversed.

As I have repeatedly acknowledged, prescribing lower carbohydrate diets in free-living subjects generally leads to greater loss of weight and body fat over the short-term when people are likely adhering most closely to the diet prescriptions. The CIM suggests that such diets offer a “metabolic advantage” that substantially increases energy expenditure and body fat loss even if diet calories are equal. However, inpatient controlled feeding studies do not support this contention as they have repeatedly failed to show significant differences in energy expenditure and body fat. Furthermore, such studies have occasionally measured significant differences in diametrically opposite directions than were predicted on the basis of carbohydrate intake and insulin secretion. These apparent falsifications of the CIM do not imply that dietary carbohydrates and insulin are unimportant for energy expenditure and body fat regulation. Rather, their role is more complicated than the CIM suggests and the model requires thoughtful modification.

On 2017 Feb 03, DAVID LUDWIG commented:

In his comment of January 31, 2017, Hall presses an extreme argument, that he successfully "falsified" major aspects of the Carbohydrate-Insulin Model (CIM) of obesity, and complains that opponents won't embrace their error. His argument boils down to 3 points:

First, Hall’s small 6-day study and his small, “observational,” “pilot” study are fundamentally correct. Regarding the 6-day study Hall KD, 2015, he continues to insist that results of a very short intervention have relevance to understanding the long-term effects of macronutrients on body composition, despite evidence that adaptation to a higher-fat diet can take at least a week and perhaps considerably longer Hawley JA, 2011 Vazquez JA, 1992 Veum VL, 2017. (We need look no further than his observational study Hall KD, 2016, to see in Figure 2B that the transient decrease in rate of fat loss upon initiation of the low-carbohydrate diet accelerates after 2 weeks.) Of note, the 36 g/d greater predicted body fat loss on his low-fat diet would, if persistent, translate into a massive advantage in adiposity after just one year. If anything, the meta-analyses of long-term clinical trials suggest the opposite Tobias DK, 2015 Mansoor N, 2016 Mancini JG, 2016 Sackner-Bernstein J, 2015 Bueno NB, 2013. Furthermore, Hall’s two studies are mutually inconsistent: The 6-day study implies a major increase in energy expenditure from fat oxidation on the low-fat diet, whereas the observational study shows an increase in energy expenditure after 2 weeks (by doubly-labeled water) on the low-carbohydrate diet. Other limitations of his observational study have been considered elsewhere.

Second, our randomized 3-arm cross-over study Ebbeling CB, 2012 is fundamentally wrong. I’ve addressed Hall’s concerns elsewhere. Here, he reiterates that the 10% difference in protein content (intended by design to reflect the Atkins diet) could account for our observed 325 kcal/d difference in energy expenditure. However, there is no basis in the literature for this belief. Among 10 studies published at the time of our feeding trial in which protein intake was compared within the physiological range (10 to 35% of total energy), energy expenditure on the higher vs. lower protein diets ranged from +95 kcal/d to -97 kcal/d, with the mean difference of near zero Dulloo AG, 1999 Hochstenbach-Waelen A, 2009 Lejeune MP, 2006 Luscombe ND, 2003 Mikkelsen PB, 2000 Veldhorst MA, 2009 Veldhorst MA, 2010 Westerterp KR, 1999 Westerterp-Plantenga MS, 2009 Whitehead JM, 1996. Though these studies have methodological limitations themselves, the finding is consistent with thermodynamic considerations that indicate a very minor increment in the "thermic effect of food" from a 10% increase in protein.

Third, 18 other studies provide definitive support his position. This facile contention disregards that these studies are riddled with the same inherent limitations as his studies, including a combination of short duration, highly limited power, indirect measurements of body composition, reliance on metabolic chambers (which have been shown to underestimate adaptive thermogenesis compared to doubly-labeled water Rosenbaum M, 1996), quality control concerns and other issues. Of the cited studies, six were 1 to 4 days Astrup A, 1994 Dirlewanger M, 2000 Davy KP, 2001 Smith SR, 2000 Thearle MS, 2013 Yerboeket-van de Venne WP, 1996, seven were 7 to 15 days Horton TJ, 1995 Shepard TY, 2001 Eckel RH, 2006 Hill JO, 1991 Schrauwen P, 1997 Treuth MS, 2003 Yang MU, 1976, and just five were 4 to 6 weeks. One of these longer studies was based on recovered data from about 30 years prior to publication, with no direct measurements of body composition or energy expenditure Leibel RL, 1992. The other four longer studies employed severe calorie restriction, which would plausibly obscure macronutrient effects over this short duration. Two of these studies had just 4 subjects per diet group Rumpler WV, 1991 Bogardus C, 1981. The remaining two showed either a non-significant (2 kg lower total body fat) Golay A, 1996 or significant (30 cc lower visceral fat) Miyashita Y, 2004 advantage for the lower-carbohydrate diet. We’ve been down this road before, with the launch of the 40-year low-fat diet era based on over-interpretation of methodologically limited research. Let’s not make the same mistake again.

Even as he over-interprets the short-term feeding studies, Hall disregards extensive animal research, high quality observational studies, mechanistic studies, and clinical trials in support of CIM, as summarized here and elsewhere Ludwig DS, 2014 Lucan SC, 2015 Templeman NM, 2017.

Finally, Hall claims that I misunderstand the notion of “energy gap.” As both Hall and I Katan MB, 2010 have considered elsewhere, a decrease in energy intake produces a compensatory decrease in energy expenditure, resulting in less weight loss than would be predicted from the simple observation that a pound of fat contains 3500 kcal. However, here we consider the opposite phenomenon – an increase in energy expenditure resulting from changing dietary quality, not quantity. There is no reason to believe that compensatory increases in energy intake would occur as a result of faster metabolic rate over a similar time frame as that observed with compensatory changes to energy restriction. (Indeed, Hall himself acknowledges the possibility that low-carbohydrate diets might also lower energy intake.) Of course, progressive weight loss regardless of cause would eventually reduce energy expenditure, but we cannot infer from current data when that energy gap would reach zero. Even with conventional assumptions, NIDDK’s Body Weight Planner indicates the 150 kcal/d change in energy balance Hall found on the low-carbohydrate diet by doubly-labeled water would produce more than a 15 lb weight loss for a typical individual over several years – amounting to half the mean change in weight that occurred during the obesity epidemic in the U.S. Why would we dismiss findings with such major potential public health significance?

Hall's premature claims of (at least partial) victory and calls for curtailment of funding for more research Freedhoff Y, 2016 do not do justice to a complicated scientific question. In view of the failure of conventional obesity treatment and the massive public health challenges, all participants in this debate would do well to acknowledge the limitations of existing evidence and join in the design of more definitive research.

On 2017 Jan 31, Kevin Hall commented:

Science progresses through an iterative process of formulating models to explain our observations and subjecting those models to experimental interrogation. A single valid experimental result that runs counter to a model prediction falsifies the model and thereby requires its reformulation. Alternatively, refutation of an apparent model falsification requires demonstrating that the experimental observation was invalid.

My review of the carbohydrate-insulin model (CIM) presented a synthesis of the evidence from 20 inpatient controlled feeding studies strongly suggesting that at least some important aspects of the model are in need of modification. In particular, our recent studies Hall KD, 2015, Hall KD, 2016 employing carefully controlled inpatient isocaloric diets with constant protein, but differing in carbohydrate and fat, resulted in statistically significant differences between the diets regarding body fat and energy expenditure that were in directions opposite to predictions of the CIM.

Rather than using our experimental results as the basis for clarifying and reformulating the CIM, Ludwig challenges their validity and simply ignores the 18 other inpatient controlled feeding studies with their conforming body of results failing to support the energy expenditure or body fat predictions of the CIM.

Ludwig’s comments on the diets used in Hall KD, 2015 are irrelevant to whether they resulted in a valid test of the CIM predictions. We fed people diets that selectively reduced 30% of baseline calories solely by restricting either carbohydrate or fat. These diets achieved substantial differences in daily insulin secretion as measured by ~20% lower 24hr urinary C-peptide excretion with the reduced carbohydrate diet as compared with the reduced fat diet (p= 0.001) which was unchanged from baseline. Whereas the reduced fat diet resulted in no significant energy expenditure changes from baseline, carbohydrate restriction resulted in a ~100 kcal/d decrease in both daily energy expenditure and sleeping metabolic rate. These results were in direct opposition to the CIM predictions, but in accord with the previous studies described in the review as well as a subsequent study demonstrating that lower insulin secretion was associated with a greater reduction of metabolic rate during weight loss Muller MJ, 2015.

Ludwig erroneously claims that the study suffered from an “inability to directly document change in fat mass by DXA”, but DXA measurements indicated statistically significant reductions in body fat with both diets. While DXA was not sufficiently precise to detect significant differences between the diets, even this null result runs counter to the predicted greater body fat loss with the reduced carbohydrate diet. Importantly, the highly sensitive fat balance technique demonstrated small but statistically significant differences in cumulative body fat loss (p<0.0001) in the direction opposite to the CIM predictions. Ludwig claims that our results are invalid because “rates of fat oxidation, the primary endpoint, are exquisitely sensitive to energy balance. A miscalculation of available energy for each diet of 5% in opposite directions could explain the study’s findings.” However, it is highly implausible that small uncertainties in the metabolizable energy content of the diet amounting to <100 kcal/d could explain the >400 kcal/d (p<0.0001) measured difference in daily fat oxidation rate. Furthermore, our results were robust to the study errors and exclusions fully reported in Hall KD, 2015 and clearly falsified important aspects of the CIM.

We previously responded Hall KD, 2016b to Ludwig’s comments Ludwig DS, 2016 on our ketogenic diet study. Ludwig now argues that we set the bar too high regarding the energy expenditure predictions of the CIM based on “speculative claims by non-scientists like Robert Atkins”. But scientists well-known for promoting low carb diets have claimed that “very low carbohydrate diets, in their early phases, also must supply substantial glucose to the brain from gluconeogenesis…the energy cost, at 4–5 kcal/gram could amount to as much as 400–600 kcal/day” Fine EJ, 2004. Ludwig also sets the energy expenditure bar quite high in his New York Times opinion article, JAMA commentary, and book “Always Hungry” where he claims to have demonstrated a 325 kcal/d increase in expenditure in accordance with the CIM predictions Ebbeling CB, 2012. What Ludwig fails to mention is that such an interpretation is confounded by the low-carbohydrate diet having 50% greater dietary protein which is well-known to increase expenditure. Ludwig also doesn’t mention that his study failed to demonstrate a significant effect on either resting or daily energy expenditure when comparing diets with the same protein content, but varying in carbohydrate and fat.

What was the energy expenditure bar set by our ketogenic diet study Hall KD, 2016? The clinical protocol specified that the primary daily energy expenditure outcome (measured by room calorimetry) must increase by >150 kcal/d to be considered physiologically meaningful. With the agreement of funders at the Nutrition Science Initiative, notable proponents of the CIM, the pre-specified 150 kcal/d threshold was used to calculate number of study subjects required to estimate the energy expenditure effect size in a homogeneous population of men consuming an extremely low carbohydrate diet. If the measured effect size exceeded 150 kcal/d then the results could be reasonably interpreted as a physiologically important increase in energy expenditure worthy of future study in a wider population using a more realistic and sustainable diets. Unfortunately, the primary energy expenditure outcome was substantially less than 150 kcal/d and it would have been unethical to retrospectively “move the goal posts” or emphasize exploratory outcomes that could possibly be interpreted as more favorable to the CIM.

Ludwig sets the bar far too low when he claims that a ~100 kcal/d effect size “would be of major scientific and clinical significance” for treatment of obesity. Ludwig bases this claim on a misunderstanding of the tiny “energy imbalance gap” between calorie intake and expenditure corresponding with the rise of population obesity prevalence Hall KD, 2011. This is especially puzzling since Ludwig himself used the same mathematical model calculations to conclude that development of obesity in adults requires an increased energy intake (or decreased expenditure) amounting to ~400-700 kcal/d Katan MB, 2010.

As described in my review, the carbohydrate-insulin model is clearly in need of reformulation regarding the predicted effects of isocaloric variations in dietary carbohydrate and fat on energy expenditure and body fat. However, other aspects of the model remain to be adequately investigated and reasonable ad hoc modifications of the model have been proposed. Finally, it is important to emphasize that regardless of whether the carbohydrate-insulin model is true or false, dietary carbohydrates and insulin may promote obesity and low carbohydrate diets may offer benefits for weight loss and metabolic health.

On 2017 Jan 17, DAVID LUDWIG commented:

In this review, Hall claims to have “falsified” the Carbohydrate-Insulin Model (CIM) of obesity as iterated by Mark Friedman and me in 2014 Ludwig DS, 2014. Hall describes this achievement as “rare” in nutritional science and analogous to the refutation of the “luminiferous ether” hypothesis of the 19th century. Elsewhere, he argues that the published data are so definitive as to warrant curtailment of further funding for macronutrient-focused obesity research Freedhoff Y, 2016. 

To loosely paraphrase Mark Twain, rumors of CIM’s demise have been greatly exaggerated.

Hall bases his case mainly on his two feeding studies, one small and short (6 days), the other small, non-randomized (i.e., observational) and designated a pilot.

In the discussion section of the 6-day study Hall KD, 2015, Hall and colleagues write: “Our relatively short-term experimental study has obvious limitations in its ability to translate to fat mass changes over prolonged durations” (NB: it can take the body weeks to fully adapt to a high fat diet Hawley JA, 2011 Vazquez JA, 1992 Veum VL, 2017). This appropriately cautious interpretation was evidently abandoned in the current review. Indeed, the study has numerous limitations beyond short duration, as reviewed elsewhere, including: 1) inability to directly document change in fat mass by DXA; 2) use of an exceptionally low fat content for the low-fat diet (< 8% of total energy), arguably without precedent in any population consuming natural diets; 3) use of a relatively mild restriction of carbohydrate (30% of total energy), well short of typical very-low-carbohydrate diets; and 4) experimental errors and exclusions of data that could confound findings. In addition, the investigators failed to verify biologically available energy of the diet (e.g., by analysis of the diets and stools for energy content). Rates of fat oxidation, the primary endpoint, are exquisitely sensitive to energy balance. A miscalculation of available energy for each diet of 5% in opposite directions could explain the study’s findings – and this possibility can’t be ruled out in studies of such short duration.

Hall’s non-randomized pilot Hall KD, 2016 potentially suffers from all the well-recognized limitations of small observational studies, importantly including confounding by any time-varying covariate. One such factor is miscalculation of energy requirements, leading to progressive weight loss that would have introduced bias against the very-low-carbohydrate diet. Other major design and interpretive limitations have been considered elsewhere.

Furthermore, Hall sets the bar for the CIM unrealistically high (i.e., 400 to 600 kcal/d greater total energy expenditure), citing speculative claims by non-scientists like Robert Atkins. In fact, effect estimates of 100 to 300 kcal/day – as demonstrated by Hall himself Hall KD, 2016 and by us Ebbeling CB, 2012 using doubly-labeled water – would be of major scientific and clinical significance if real, and do not represent "ad hoc modifications" to evade "falsification." (For comparison, Hall previously argued that the actual energy imbalance underlying the entire obesity epidemic is < 10 kcal/d Hall KD, 2011.)

To test the CIM, we need high-quality studies of adequate duration to eliminate transient biological processes (ideally ≥ 1 month); using a randomized-controlled design; with definitive measurements of body composition (e.g. DXA or MRI); and including appropriate process measures to assure that the diets are properly controlled for biologically available energy content. No such studies have yet been published. Thus, the CIM is neither proven nor “falsified” by existing data. In view of the complexity of diet, many high-quality studies will likely be needed to provide a complete answer to this question, versions of which have been debated for a century.

The CIM aims to explain a paradox: Body weight is controlled (“defended”) by biological factors affecting fat storage, hunger and energy expenditure Leibel RL, 1995. However, the average defended body weight has increased rapidly throughout the world among genetically stable populations. Lacking a definitive explanation for the ongoing obesity epidemic, or effective non-surgical treatment, we should not casually dismiss CIM, especially in light of many studies suggesting benefits of carbohydrate-modified/higher-fat diets for obesity Tobias DK, 2015 Mansoor N, 2016 Mancini JG, 2016 Sackner-Bernstein J, 2015 Bueno NB, 2013, cardiovascular disease Estruch R, 2013 and possibly longevity Wang DD, 2016.

On 2017 Dec 05, Alex Vasquez commented:

Apparently the publisher is not connecting these related publications for appropriate context; see: Vasquez A. Correspondence regarding Cutshall, Bergstrom, Kalish's "Evaluation of a functional medicine approach to treating fatigue, stress, and digestive issues in women.” Complement Ther Clin Pract. 2016 Oct 19 https://doi.org/10.1016/j.ctcp.2016.10.001

On 2017 Jul 27, Robin W.M. Vernooij commented:

We have developed a fillable, user-friendly PDF version of CheckUp, which can be found at the EQUATOR ( Enhancing the QUAlity and Transparency Of health Research ) Library (http://www.equator-network.org/reporting-guidelines/reporting-items-for-updated-clinical-guidelines-checkup/).

CheckUp has recently been translated into Spanish and Dutch (Chinese and Czech versions are being prepared); these translated versions can also be found at the EQUATOR library. Researchers are invited to translate CheckUp into other languages.

On 2017 Jul 01, Lydia Maniatis commented:

"Could these null findings result simply from poor data quality in infants?"

That a study even warrants such a statement implies a lack of theoretical and methodological rigor. Such questions cannot be resolved post hoc - experiments need to be planned so as to avoid them altogether. The authors feel that "Several observations argue against this [poor quality data] interpretation," but such special pleading by the authors doesn't make me feel any better.

This is a study in which the conceptual categories are crude - e.g. "scenes" is a category - calling into question its replicability (given the broad latitude in selecting stimuli we could label "scenes.") Post hoc evaluations of data - model-fitting, etc - are also poor practice. All they can do is describe a particular dataset, confounds and all. Because the authors make no predictions, emphasizing instead the relative novelty of their technique, one might overlook the fact that data generated without a clear theoretical premise guiding control of variables/potential confounds is of very limited theoretical value. Basically, they're just playing with their toys.

Despite what I see as poor scientific practice, I don't think we needed an fMRI study to to "suggest" to us that, by 4–6 months, babies can distinguish "faces" from "scenes."

On 2017 Apr 04, Randi Pechacek commented:

Aaron Weimann, the 1st author on this paper, wrote a blog post on microBEnet briefly discussing this new software. Read about it here.

On 2017 Mar 09, Atanas G. Atanasov commented:

Very interesting and relevant topic. I have featured this publication at: http://healthandscienceportal.blogspot.com/2017/03/role-of-intestinal-microbiota-and.html

On 2017 Jan 07, Lydia Maniatis commented:

“In conclusion, using a psychophysical method, for the first time we showed that the timescale of adaption mechanisms for the mid-level visual areas were substantially slower than those for the early visual areas.”

Psychophysical methods are amazing. They let you tap into specific levels of the brain, just by requiring observers to press one of two buttons. As you can imagine, some heavy-duty theoretical and empirical preparation has gone into laying the ground for such a simple but penetrating method.

One example of this preparation is the assertion by Graham (1992) that under certain “simple” conditions, the brain becomes transparent, so that the percept is a direct reflection of, e.g. the activity of V1 neurons. (Like bright students, the higher levels can’t be bothered to respond to very boring stimulation). She concluded this after a subset of a vast number of experiments performed in the very active field had proven “consistent” with the “classical” view of V1 behavior, at a time when V1 was thought to be pretty much all there was (for vision). (The “classical” view was later shown to be premature and inadequate, making the achievement of consistency in this body of work even more impressive). If one wanted to be ornery, one might compare Graham’s position to saying that we can drop an object into a Rube Goldberg contraption and trigger only the first event in the series, while the other events simply disengage, due to the simplicity of the object – perhaps a simple, sinusoidal surface. To be fair, though, the visual system is not as integrated or complex as those darned contraptions.

The incorporation of this type of syllogism into the interpretation of psychophysical data was duly noted by Teller (1984), who, impressed, dubbed it the “nothing mucks it up proviso.” It has obviously remained a pillar of psychophysical research, then and now.

The other important proviso is the assumption that the visual system performs a Fourier analysis, or a system of little Fourier analyses, or something, on the image. There is no evidence for or logic to this proviso (e.g. no imaginable functional reason or even remotely adequate practical account), but, in conjunction with the transparency assumption, it becomes a very powerful tool: little sinusoidal patches tap directly into particular neural populations, or “spatial filters,” whose activity may be observed via a perceiving subject’s button tap (and a few dozen other “linking propositions,” methodological choices and number-crunching/modeling choices for which we have to consult each study individually). (There are also certain minor logical problems with the notion of “detectors,” a concept invoked in the present paper; interested readers should consult Teller (1984))

The basic theoretical ground has been so thoroughly packed that there is little reason for authors to explain their rationale before launching into their methods and results. The gist of the matter, as indicated in the brief introduction, is that Hancock and Pierce (2008) “proposed that the exposure to the compound [grating] pattern gave rise to more adaptation in the mid-level visual areas (e.g., V4) than the exposure to the component gratings.” Hancock and Pierce (2008) doubtless had a good reason for so proposing. Mei et al (2017) extend these proposals, via more gratings, and button presses, to generate even more penetrating proposals. These may become practically testable at some point in the distant future; the rationale, as mentioned, is already well-developed.

n.b. Due to transparency considerations, results apply to gratings only, either individual or overlapping.

On 2017 May 25, Lydia Maniatis commented:

Comment 2:Below are some of the assumptions entailed by the sixty-year-old "signal detection theory," as described by Nevin (1969) in a review of Green and Swets (1966), the founding text of sdt.

"Signal detection theory [has proposed] an indirectly derived measure of sensitivity...This measure is defined as the separation...between a pair of hypothesized normal density functions representing the internally observed effects of signal plus noise, an noise alone."

In other words, for any image an investigator might present, the nervous system of the observer generates a pair of probability functions related to the presence of absence of a feature of that image that the investigator has in mind and which he/she has instructed the observer to watch for. The observer perceives this feature on the basis of some form of knowledge of these functions. These functions have no perceptual correlate, nor is the observer aware of them, nor is there any explanation of how or why they would be represented at the neural level.

"The subject's pre-experimental biases, his expectations based on instructions and the a priori probability of signal, and the effects of the consequences of responding, are all subsumed under the parameter beta. The subject is assumed to transform his observations into a likelihood ratio, which is the ratio of the probability density of an observation if a signal is present to the probability density of that observation in the absence of signal. He is assumed, further, to partition the likelihood ratio continuum so that one response occurs if the likelihood ratio exceeds beta, and the other if it is less than beta."

Wow. None of these assumptions have any relationship to perceptual experience. Are they in the least plausible, or in any conceivable way testable? They underlie much of the data collection in contemporary vision science. They are dutifully taught by instructors; learning such material clearly requires that students set aside any critical thinking instincts.

The chief impetus behind SDT seems to have been a desire for mathematical neatness, rather than for the achievement of insight and discovery.

On 2017 May 23, Lydia Maniatis commented:

"For over 60 years, signal detection theory has been used to analyze detection and discrimination tasks. Typically, sensory data are assumed to be Gaussian with equal variances but different means for signal-absent and signal-present trials. To decide, the observer compares the noisy sensory data to a fixed decision criterion. Performance is summarized by d0 (discriminability) and c (decision criterion) based on measured hit and false-alarm rates."

What should be noted about the above excerpt is the way in which a statement of historical fact is offered as a substitute for a rationale. What scientist could argue with a 60-year-old practice?

The "typical" assumptions are not credible, but if the authors believe in them, it would be great if they could propose a way to test them, as well as work out arguments against the seemingly insurmountable objections to treating neurons as detectors, objections raise by, for example, Teller (1984).

While they are at it, they might explain what they mean by "sensory data." Are they referring to the reaction of a single photoreceptor when struck by a photon of a particular wavelength and intensity? Or to one of the infinitely variable combinations of photon intensities/wavelengths hitting the entire retina at any given moment - combinations which mediate what is perceived at any local point in the visual field? How do we get a Gaussian distribution when every passing state of the whole retina, and even parts of it, is more than likely unique? When, with eyes open, is the visual system in a "signal-absent" state?

There is clearly a perfect confusion here about the "decision" by the visual process that produces the conscious percept and the decision by the conscious observer trying to recall and compare percepts presented under suboptimal conditions (very brief presentation times) and decide whether they conform to an extrinsic criterion. (What is the logic of the brief presentation? And why muddy the waters with forced choices? (I suspect it's to ensure the necessary "noisiness" of results)).

"For 5 out of 10 observers in the covert-criterion task, the exponentially weighted movingaverage model fit the best. Of the remaining five observers, one was fit equally well by the exponentially weighted moving-average and the limited-memory models, one was fit best by the Bayesian selection, exponentially weighted moving-average, and the reinforcement learning models, one was fit best by the Bayesian selection and the reinforcement learning models, one was fit best by the exponentially weighted moving-average and reinforcement learning models, and one was best fit by the reinforcement learning model. At the group level, the exceedance probability for the exponentially weighted moving-average is very high (慸ponential = .95) suggesting that given the group data, it is a more likely model than the alternatives (Table 1). In the overt-criterion task, the exponentially weighted moving-average model fit best for 5 out of 10 observers. Of the remaining five observers, one was fit equally well by the exponentially weighted moving-average and the reinforcement learning models, two were fit best by the reinforcement-learning model, and two were fit best by the limited-memory model. At the group level, the exceedance probability for the exponentially weighted moving-average model (慸ponential = .78) is higher than the alternatives suggesting that it is more likely given the group data (Table 2)."

Note how, in the modern conception of vision science practice, failure is not an option; the criterion is simply which of a number of arbitrary models "fits best," overall. Inconsistency with experiment is not cause to reject a "model", as long as other models did worse or did as well, but in fewer case.

What is the aim here?

On 2017 Aug 04, Henri de la Salle commented:

This review privileges the view that mRNAs are translated in platelets. However, the biological function of mRNAs in platelets is not so clear. Our works do not agree with the conclusions drawn from other ones quoted in this review. We have demonstrated (Angénieux et al., PLoS One. 2016 Jan 25;11(1):e0148064. doi: 10.1371/journal.pone.0148064) that the lifespan of mRNAs and rRNAs in platelets is reduced, only a few hours. Accordingly, the translation activity in platelets rapidly decay, within a few hours. Thus, in vivo, translation of non-mitochondrial mRNAs only occurs in young platelets, a few percent of blood platelets in physiologic conditions. Most of the works reporting translation in platelets should be revisited by quantifying the number of transcripts of interest actually present in platelets. The RNAscope technique is powerful to investigate this problem; our works indicated that the most frequent transcript (eg beta actin mRNA) can be detected most if not all young platelets, but in only few percent of total blood platelets in homeostatic conditions. Finally, the biological role of translation in young platelets need to be established using accurate quantification methods, which is not easy with these cells.

On 2017 Dec 26, Elena Gavrilova commented:

The initial article with the experimental study could be found here: https://www.ncbi.nlm.nih.gov/pubmed/27769099 The current article provides a detailed response to E.V. Dueva’s concerns regarding the experimental study. Moreover, the current article already contains response to the concerns raised in E.V. Dueva’s comment. Both our articles are fully transparent for the readers so they have an opportunity to familiarize themselves with the study results and detailed response to the concerns and make their own opinion.

On 2017 Jul 21, Hans Bisgaard commented:

Thank you for your interest in our study. We will be pleased to address any questions or comments in the proper scientific manner, where you submit these to the journal as a Letter to the Editor.

Sincerely

Hans Bisgaard, Bo Chawes, Jakob Stokholm and Klaus Bønnelykke COPSAC, Copenhagen, Denmark

On 2017 Jun 29, Martijn Katan commented:

My colleague Paul Brand and I published a Dutch-language comment on this paper in the Dutch Medical Journal. The English abstract is below and on www.ncbi.nlm.nih.gov/pubmed/28635579.

Abstract: Taking fish oil supplements in the third trimester of pregnancy was associated with significantly less wheezing or asthma in the child at the age of 3-5 years, according to a randomized clinical trial by Bisgaard et al., NEJM 2017. However, the results of this study should be interpreted with caution. The primary end points were modified at a late stage in the study, and two primary end points, eczema in the first 3 years of life and allergic sensitization at 18 months of age, were demoted to secondary end points, and showed no significant effect of treatment. Furthermore, the age range for the published primary end point, persistent wheeze, differed from that in the protocol. Additional concerns include the emphasis on outcomes by omega-3 fatty acid levels in the blood, a post hoc subgroup analysis not included in the protocol. In our opinion, this study does not justify advising routine fish oil supplements in pregnancy.

On date unavailable, commented:

None

On 2017 Jul 25, Konstantinos Fountoulakis commented:

Nice way to reply without replying to exact and specific questions. You already know that the NEJM editor rejected a letter by me and as i can see here he has also rejected other similar letters which raised the same questions. These specific questions seem to burn and i again mention them here:

1. Did you or you did not change the primary outcome after registering the trail and during the study, and after the results of some of the subjects were available? (not in my comments but it needs a definite answer which i did not see so far)<br>
2. Did you or you did not include in the paper a different primary outcome (3-5 years) from what you had eventually registered in the protocol (0-3 years) and specifically stated in the paper that this was the primary outcome of the study? Is 0-3 identical to 3-5?

Well i have no way of publishing this as a letter to the editor, I have already tried. To make things worse, the reply letter says (verbatim) that 'As clearly stated in the article the primary outcome was extended to distinguish wheezing children from asthmatic children'. I hope you will respond to the above issues and clarify once and for all the problem.

On 2017 Jul 21, Hans Bisgaard commented:

Thank you for your interest in our study. We will be pleased to address any questions or comments in the proper scientific manner, where you submit these to the journal as a Letter to the Editor.

Sincerely

Hans Bisgaard, Bo Chawes, Jakob Stokholm and Klaus Bønnelykke COPSAC, Copenhagen, Denmark

On 2017 Apr 20, Konstantinos Fountoulakis commented:

You do not like my critisism, and your reply is insulting. However in https://clinicaltrials.gov/ct2/show/NCT00798226 it says verbatim: 'Primary Outcome Measures: Persistent wheeze 0 to 3 years of age [ Time Frame: 3 years ]

in the paper you say (again verbatim): Primary End Point During the prespecified, double-blind follow-up period, which covered children from birth to between 3 and 5 years of age, 136 of 695 children (19.6%) received a diagnosis of persistent wheeze or asthma, and this condition was associated with reduced lung function by 5 years of age, with parental asthma, and with a genetic risk of asthma

It is quite different 0 to 3 and 0-to between 3 and 5

I would appreciate an answer on this. BTW Facebook is also good in disceminating scientific findings