On 2016 Dec 23, Clive Bates commented:

In their published reply to this article, Endothelial progenitor cell release is usually considered a beneficial effect: Problems in interpreting the acute effects of e-cigarette use, Farsalinos and Polosa point out that the measured increase in endothelial progenitor cells (EPCs) is usually associated with beneficial effects, and not necessarily a cause for the concerns expressed by the authors.

Farsalinos and Polosa point out that several problem conditions are associated with lower EPC levels:

However, the increase in EPC levels is largely interpreted in the scientific literature as a beneficial effect while a reduction is interpreted as an adverse prognostic marker. Several risk factors for cardiovascular disease, such as ageing, hyperlipidemia, hypertension, obesity and diabetes, are associated with reduced levels and functional impairment of EPCs. Similar associations were found with “non-classic” risk factors such as high C-reactive protein and homocysteine, and low vitamin D levels. Smokers have lower levels of EPCs compared to nonsmokers.

They also point out that many positive conditions are associated with higher EPCs:

Various short-term or acute interventions are associated with elevated EPCs. Consumption of red wine, switching to Mediterranean diet and acute exercise are associated with elevated number of circulating EPCs in healthy subjects. EPCs increase shortly after smoking cessation (especially in light smokers), with nicotine patch users having slightly higher (but not statistically significant) elevation in EPCs after smoking cessation compared to non-users. Short-term administration of green tea also caused an increase in EPCs in young healthy smokers. In all the above-mentioned interventions, the increase in EPCs was interpreted as a beneficial effect and there was no suggestion that it was a response to vascular injury caused by the intervention

This is merely the latest of several recent analyses in which observations of acute effects of nicotine have been uncritically assumed to be a marker for a chronic cardiovascular disease risk (see Vlachopoulos C, 2016 and Carnevale R, 2016 for example), generating alarming news headlines as a result (see: E-cigs: The incendiary truth... Just 10 puffs increases your risk of heart disease, Daily Mail, 3 Dec 2016).

Please see Benowitz NL, 2016 for a more credible and complete account of the cardiovascular effects of nicotine as they relate to e-cigarettes. Benowitz and Burbank review the relevant evidence and summarise the current state of knowledge as follows:

The cardiovascular safety of nicotine is an important question in the current debate on the benefits vs. risks of electronic cigarettes and related public health policy. Nicotine exerts pharmacologic effects that could contribute to acute cardiovascular events and accelerated atherogenesis experienced by cigarette smokers. Studies of nicotine medications and smokeless tobacco indicate that the risks of nicotine without tobacco combustion products (cigarette smoke) are low compared to cigarette smoking, but are still of concern in people with cardiovascular disease. Electronic cigarettes deliver nicotine without combustion of tobacco and appear to pose low-cardiovascular risk, at least with short-term use, in healthy users.

The absence of serious disease risk when nicotine is consumed through NRT or smokeless tobacco (i.e. without the products of combustion of tobacco leaf) should be a basis for reassuring and encouraging smokers considering switching to vaping.

I hope the authors and journal will take care that any misunderstandings generated by their work and the media attention that followed will be corrected and placed in context. The problem is that alarming but baseless statements about vaping risks can easily have the unintended effect of encouraging continued smoking and cause harm as a result.

On 2017 Aug 11, Jos Verbeek commented:

Unfortunately this review did not adhere to the following important Cochrane standards. There was no published protocol. The authors combined the results of all RCTs and case series thus artificially inflating the effect. They pooled very different interventions in one pooled result even though heterogeneity was as high as 82%. I believe that calculating one pooled effect size for interventions as different as meditation, communication skills or improved work schedules does not make sense. It is also not a methodological standard to loosely asses the quality of the evidence as moderate without further justification and not using the GRADE approach. Another interesting item is the claim that the results are ‘clinical meaningful reductions’. It is not clear what the authors refer to. With patient-reported outcomes one would be interested in a minimally clinically relevant difference or reductions that patients perceive as an improvement. However, for the Maslach Burnout Inventory this difference has never been established as far as we know. Thus we don’t know what the clinical meaning of reductions on this scale is. It is very well conceivable that these will not be perceived as improvements by an individual health care worker. West et al neither discuss the problem of small studies that is apparent with the average number of participants being less than 50 in the 15 included trials but again loosely refer to the funnel plot as not indicating publication bias. I believe that physicians are put on the wrong foot with the conclusion of moderate quality evidence of clinically meaningful reductions in burnout based on this review.

On 2016 Oct 08, Lydia Maniatis commented:

The authors state in their conclusion that: "At each contrast level, the perceived depth first increases with the magnitude of disparity modulation up to a critical value and then decreases gradually with further increases in the magnitude of disparity modulation. "

As is well-known, perceived depth is largely mediated by stimulus structure. The stimuli used by Chen et al (2016) have a structure that produces a 3D impression. Do the "single-cycle" and the "corrugated" version, produce the same depth impression as forms, i.e. viewed in outline monocularly? Then this would have to be factored in to the conclusions re disparity and contrast. Are the authors claiming that they have controlled for this factor? Then this should be stated. Otherwise, their results can only be said to hold for the particular stimuli they employed, not for perception in general. In this case, there are no principles being investigated here; results have no predictive value; conclusions are purely ad hoc.

The most simple illustration of the role of luminance structure (combined with the principles instantiated in the visual system) is what happens in a completely homogeneous visual field, produced by a flat surface - in other words one with zero contrast. In this case, we perceive a cloudy, 3D space. A small luminance variation on that surface collapses the perceived fog into a flat perceived surface. If we converted that surface into an image that we would refer to as trompe l'oeil, we would have a very strong impression of 3D structure with zero disparity. At low contrast or high contrast, if structure implies depth, we'll see depth, and vice versa.

With respect to subjects, as is very common in psychophysics, there were very few - three here - and one of them was an author. The participation of authors is odd especially in light of the fact that we're told explicitly that the other two subjects were "naive to the purpose of this study." If it's important that the observers be naive, then why is an author a subject; if it's not important, why mention it?

Other points: The authors fail to consider the distinction between luminance contrast and perceived constrast. They modulate the luminance contrast between dots and background. We know that small elements, like thin lines, tend to produce assimilation with the background, i.e. lowers perceived contrast.

The authors mention that previous studies, using various stimuli and conditions, have produced inconsistent results. Despite these studies, we're told, " it is still difficult to infer the effect of luminance contrast effect on perceived depth in a scene..." So the question the authors seem to be asking is "what is the effect of luminance contrast on perceived depth in a scene?" But if the previous studies show anything, it is that conditions matter; so a search for "the effect" seems inappropriate. Picking a set of conditions out of a hat and testing them will only tell us about the luminance contrast effect under those conditions. The set of possible conditions is infinite.

The authors state in their intro that "as shown in signal detection theory (Green & Swets, 1966; Chen & Tyler, 2001), the threshold measurement constituting stereoacuity depends not only on the intensity of the stimulus but also on the internal noise." Neither Green & Swets (1966) nor Chen & Tyler (2001) have shown that there is noise in the visual system, and they have certainly not shown that this putative internal noise is maintained and expressed in the percept. The "internal noise" claim has generated no evidence (it is not even clear what this evidence would look like, including in terms of physiological measurements), and is maintained on the basis of studies employing a narrow set of conditions generating crude datasets whose results are highly overinterpreted.

On 2016 Oct 11, Lydia Maniatis commented:

The authors of this study transparently misrepresent and/or misunderstand the theoretical situation with respect to the subject they are investigating.

The most blatant oversight involves the failure to note that the type of stimulus being used here – a checkerboard configuration – was shown decades ago to reduce, rather than enhance, perceived contrast in adjacent surfaces. (see demo here: https://www.researchgate.net/figure/225158523_fig5_Fig-5-The-DeValois-and-DeValois-Checkerboard-stimulus). This fact was similarly overlooked by Maertens, Wichmann and Shapely (2015), who simply assumed the opposite.

The checkerboard configuration is clearly a special and unusual case, in the sense that it elicits the impression of adjacent surfaces rather the more typical experience of surfaces on top of backgrounds (figure/ground relationships). It is unlikely that an ad hoc model that involves averaging of luminances would work for both the former and the latter sets of conditions.*

The Missing Segmentation Step

In general, Wiebel et al (2016) have chosen not to address the fundamental factor mediating color and lightness, i.e the structure of the stimulus and the principles by which the visual system organizes the retinal projection to form the percept. Yet Zeiner and Maertens (2014), whose “normalized contrast model” the authors are applying here, acknowledged this gap in retrospect:

“The important piece of information that is still missing, and which we secretly inserted, was the knowledge about regions of different contrast range. Here we simply used the values that we knew to originate from the checks within the regions corresponding to plain view, shadow, or transparent media, but for a model to be applicable to any image this segmentation step still needs to be elucidated.”

But the entire problem - the ability to predict perceived lightness of any surface - lies precisely in the "segmentation step." Furthermore, since they haven't taken connection of structure to assimilation and contrast into account, it is, as noted above, doubtful that their model would work in general, even if they were to similarly sneak segmentation in through the back door.

Wiebel, Singh and Maertens sidestep the issue, simply assuming “that the visual system is sensitive to differences in contrast range and can use them to detect regions seen in plain view, because they have the highest contrast range.”

The problem, of course, is that the contrast ranges of different “regions” of an image depend on how the image is divided up in the perceptual process; again, they depend on the “segmentation step” that Wiebel, Singh and Maertens, like Zeiner and Maertens, (2016) sneak in unanalyzed.

The failure to address structure and principles of organization is also reflected in the fact that their definition of contrast depends on comparing luminances in an arbitrary, local area of the total image, rather than everything the observer could see, both on and off the screen. Again, the consequences of this global image depend on the “segmentation” step.

Model Failures Can't Be Redeemed By Ad Hoc Successes

The “model” the authors leave us with is not only ad hoc, it fails tests within this study. The abstract indicates the Zeiner Maertens model model fails even for the narrow (and very unusual) set of conditions chosen, but that “model extensions” “fit the observed data well.” In the discussion we find that the fits weren’t all that good: “For the normalized contrast model, significant differences between model predictions and observed data were shown for Reflectance 6 (p < 0.05) in the light transparency. For the dark transparency, significant differences were found for Reflectances 3, 5, 6, 8, and 9 (p < 0.05).” In other words, even for the highly selective conditions used, fits were hit-and-miss. The authors don’t have the theoretical tools to explain why (though the tools are arguably available). But they speculate, and make the following very odd statement.

In contrasting their “normalized contrast model” with the “contrast ratio model,” they note that one works for better for the “light transparency” conditions and the other for the “dark transparency” conditions. This, they argue, is due to the anchoring assumptions of each model. They propose to create new stimuli to “decide between the two models.”

But each of the models has already failed. These failures won’t be undone by any future “successes.” Any such successes would obviously be ad hoc, another theoretically and factually agnostic exercise such as the one we are discussing. No amount of such structure-blind, ad hoc model-building could take us even to the point of knowledge already available, though ignored.

Short Summary: 1. The authors construct ad hoc models of lightness perception without taking into account the fundamental mediator of lightness, i.e. stimulus structure and the principles by which the visual system organizes the retinal projection.

1. Their model fails a number of tests in this study, but they propose to keep on testing it, in order to “decide” between it and an alternative, which has also failed a number of the tests put to it in this study.

It is not clear what is the purpose in testing two failed, ad hoc models. Clearly, they are both capable of “succeeding” and of failing in an infinite number of cases.

*The assimilation that we see in the checkerboard demo is linked to the fact that contrast enhancement is highly correlated with perceived figure ground relationships, making the borders of the figure more visually salient. Percieved figure-ground relationships in turn hinge on conditions indicating figural overlap, such as intersecting continuous edges of potentially closed figures. In the case of the checkerboard, the perceptual relationships between squares is one of adjacency rather than overlap. Luminance changes interpreted as occurring within a single surface tend to produce assimilation, as for example in the case of the Koffka ring.

On 2017 Jun 14, C Trolle commented:

DNAmethylation and RNAseq data from out Turner study has finally been released.

https://www.ebi.ac.uk/ega/studies/EGAS00001002190

On 2016 Sep 30, Clive Bates commented:

There is nothing at all in these findings to justify the conclusion. In fact, the findings are more likely to support the opposite - that such social media activity is helpful in reducing smoking. If vaping companies are "enticing consumers" it is almost always to stop smoking and to vape instead. So if these tweets were actually affecting behaviour, it is likely that it would be in a way that reduces smoking and is beneficial to public health.

The authors cannot know if these tweets do actually cause changes in smoking or vaping status. However, it seems likely (by which I mean 'obvious') that vaping-related tweets would be seen by almost exclusively people who already vape. The key design feature of twitter is that users opt into the content they wish to see by choosing to follow other users. Non-vapers are unlikely to follow a vaping company or vaping review feed. Equally, advertising is targeted at users algorithmically to reach users and potential users )i.e. smokers). So the likelihood is that, if these tweets have any impact at all, it will be in shaping preferences among those already vaping for particular flavours, brand or retailers or advertising an alternative to smoking, albeit one that the authors appear to disapprove of.

The authors seem concerned that only 3% mention vaping as a way of quitting smoking. Why should that matter at all? If people are attracted to vaping for different reasons but give up smoking as a result, what's the problem? Furthermore, The Vaper points claims about quitting smoking are usually banned or deemed irresponsible Kavuluru R, 2016. Other authors have managed to be worried about the finding the opposite van der Tempel J, 2016.

There is no justification for doing this work in the first place, no case to publish such flawed interpretation of the findings, and no need to spend any time or money following its self-evidently false conclusion and inappropriate policy recommendation.

Disclosure: I am a longstanding advocate for 'harm reduction' approaches to public health. I was director of Action on Smoking and Health UK from 1997-2003. I have no competing interests with respect to any of the relevant industries.

On 2016 Nov 07, John Sotos commented:

In assessing the possible impact of machine learning on clinical medicine, Obermeyer and Emanuel(1) describe the narrowing gap between human vs. computer analysis of images, and declare that "machine learning will displace much of the work of radiologists and anatomical pathologists."

We hesitate to agree, owing to the Jevons paradox and elastic demand for medical imaging.

In 1865, the economist William Jevons predicted that more efficient coal-burning in manufacturing plants would not lower the nationwide consumption of coal. Instead, the lower cost per unit of energy would increase demand for coal energy and thereby increase consumption(2).

Thus, assuming machine interpretation lowers the cost per imaging study, future human case loads will depend on the quantitative balance between a Jevonsonian increase in imaging (if any(3)) and the fraction of cases where computers completely exclude humans (e.g. only 25% for contemporary computerized Pap smear interpretation(4)).

Clearly, major changes are coming, but, given healthcare's tangled economics, it is premature to affirm that computerized image interpretation will decimate physician workloads.

John Sotos, MD

Lester Russell, BM DRCOG MRCGP MBA

(1) Obermeyer Z, Emanuel EJ. Predicting the future -- big data, machine learning, and clinical medicine. N Engl J Med. 2016; 375: 1216-1219.

(2) Jevons WS. The Coal Question. London: Macmillan and Co., 1865. Pages 102-104.

(3) Polimeni JM, Mayumi K, Giampetro M, Alcott B. The Jevons Paradox and the Myth of Resource Efficiency Improvements. New York: Earthscan Routledge, 2008.

(4) Bengtsson E, Malm P. Screening for cervical cancer using automated analysis of Pap-smears. Computational and Mathematical Methods in Medicine. 2014; Article ID 842037. http://dx.doi.org/10.1155/2014/842037

On 2016 Nov 10, Øjvind Lidegaard commented:

Thanks to Chelsea Polis for her interest in and comments on our study.

First, the group of non-users, which was the reference group in the main analysis, includes those users of copper IUD which Chelsea Polis calls for, in addition to users of barrier methods, eventually combined with natural methods such as interrupted coitus or calendar methods. The important point here is, that those women who previously have used or in the future will use hormonal contraception are among those women. Therefore, it is not correct to consider the control group as a group of women not using contraceptive methods.

Our recommendation of further studies on this issue was not primarily due to uncertainty about the methods or the results we achieved, but more the fact that sometimes scepticism is easier to overcome when several study groups reach the same results as achieved in one sound large cohort study as the Danish study.

We also made assessments including pregnant women. For all users of oral contraceptives the relative risk of first antidepressant use actually increased with inclusion of pregnant and delivering women from 1.23 (1.22-1.25) to 1.31 (1.29-1.32) and for the 15-19 year old users the relative risk of antidepressant use was unchanged 1.8 (1.75-1.84). The increase is due to those women who begin taking oral contraceptives within the first six months after delivery, and who already have an increased risk of depression due to their delivery.

In Denmark 4% of women of reproductive age are pregnant, while 40% are current users of some kind of hormonal contraception. That is another good reason why the inclusion of pregnant women does not have much impact on the risk of depression in users of hormonal contraception. And remember that the majority of delivering women get pregnant because they want to, and not because of contraceptive failure. Women getting unwanted pregnant and choose to terminate their pregnancy, generally do not get depressed, which was demonstrated in another large Danish prospective study (1), despite the frequent claim of the opposite from especially opponents of legal abortions.

The biggest weakness of our study is in our opinion the comparison group of non-users in the main analysis. A more correct comparison group would have been never-users. The influence of hormonal contraception on depression risk increases from 1.2 to 1.7 with this change in comparison group. So if anything, our relative risk figures are underestimated.

1. Munk-Olsen T, Laursen TM, Pedersen CB, Lidegaard Ø, Mortensen PB. Induced first-trimester abortion and risk of mental disorder. NEJM 2011; 364; 332-9.

On 2016 Nov 07, Chelsea Polis commented:

This analysis by Skovlund et al. (1) suggests that Danish women who currently or recently used various types of hormonal contraception may be at greater risk of being diagnosed with depression or initiating use of antidepressants, as compared against those who formerly or never used hormonal contraception. As the authors conclude, this study suggests that “further studies are warranted to examine depression as a potential adverse effect of hormonal contraceptive use”. This is particularly important since the study was unable to provide information on these outcomes among women using non-hormonal contraceptive methods, such as a copper IUD, which may have helped to clarify whether the observed associations were related to factors common to women choosing to use contraception, or to the hormonal content of the methods assessed.

Importantly, the investigators note that they censored person-time during pregnancy and through six months post-partum. The authors characterize this as a strength of the study, noting that it was done to reduce the influence of postpartum depression on the results. However, women not using highly effective methods of contraception are presumably more likely to become unintentionally pregnant, which also has implications for women’s mental health. (2)

A sensitivity analysis not excluding pregnant and post-partum person-time could be useful in better understanding the potential competing risks faced by women in their day-to-day lives. It would be helpful for the authors to present pregnancy rates by contraceptive status, and to replicate the main analyses without excluding pregnant and post-partum person-time.

Sincerely,

Chelsea B. Polis, PhD, Senior Research Scientist, Guttmacher Institute

Ruth B. Merkatz, PhD, RN, FAAN, Director, Population Council

1. Skovlund CW, Mørch LS, Kessing LV, Lidegaard Ø. Association of hormonal contraception with depression. JAMA Psychiatry 2016 (Epub ahead of print).
2. Abajobir AA, Maravilla JC, Alati R, Najman JM. A systematic review and meta-analysis of the association between unintended pregnancy and perinatal depression. J Affect Disord 2016;192:56-63.

On 2016 Oct 04, Quinn Capers commented:

Thank you for your interest in our work. Briefly, we are aware of theories that the IAT may measure something other than racial bias. However, regardless of what they are actually measuring, IAT results predict discriminatory behavior. Secondly, it was beyond the scope of the paper to control for the Hawthorne effect, but we acknowledge that behaviors could have changed because committee members were aware that they were being observed. Finally, we agree that an analysis of the composition of the class following the IAT should be accompanied by full admissions statistics pre- and post- the exercise. This is provided in the paper and commented on (Table 2). Word count restrictions prevented us from including this information in the abstract. Feel free to email me after reading the full length paper.

On 2016 Sep 30, Thomas Heston commented:

Does the black-white implicit association test measures racial bias? Or something else? (Kaufman SB. Psych Today 28-Jan-2011 https://goo.gl/h3jvw1). Suffice it to say that there does exist at least some controversy. The authors also failed to control for the Hawthorne Effect (BMJ 2015;351:h4672 http://bit.ly/2dqvD4p). The statement that the class that matriculated following the IAT exercise was the most diverse is really meaningless without examining the applicant pool. Examine bias? Yes, by all means. But don't ignore potential test bias and researcher bias which may make the results unscientific.

On 2017 Oct 26, Martine Crasnier-Mednansky commented:

This paper reinforces –perhaps validates– the authors’ previous work (Mondal M, 2014) indicating PTS-transported GlcNAc is utilized by Vibrio cholerae in the mucus layer, as further explained.

Data in Meibom KL, 2004, particularly supplemental figure 7, clearly indicate chiA2 (VCA0027) is not upregulated by GlcNAc. Therefore, in agreement with the present work, GlcNAc utilization by V. cholerae in the mucus may rely on the periplasmic activation of ChiS for production of ChiA2. Because chiS mutant strains do not produce extracellular chitinases in the presence of chitin oligomers (Li X, 2004), they are unlikely to produce chitinases in the presence of mucin (the authors report ChiS is activated in the presence of mucin). Thus, both chiA2 and chiS mutant strains may prevent colonization of the intestine because they are both unable to cause mucin hydrolysis by ChiA2 and subsequent release of GlcNAc, which, according to the authors’ original 2014 proposal, is necessary for growth and survival in the mucus. The mucin-derived 'inducer' for ChiS activation is possibly (GlcNAc)3, as the authors reported (GlcNAc)3 is released upon mucin hydrolysis by ChiA2.

On 2016 Sep 28, Jonathan Eisen commented:

A preprint of this paper is available at BioRXiv: http://biorxiv.org/content/early/2016/08/25/071548

On 2017 Jun 12, Maria Sammartino commented:

"The interaction of the electron beam, emitted by the gun, with the sample induces an excitation, energy is lost and a single X-ray is emitted that is characteristic of the element hit." A very bad description of the EDS! Anyway the author Gatti A.M. improved her knowledge on the subject; really in one of her oldest article ( Liver and kidney foreign bodies granulomatosis in a patient with malocclusion, bruxism, and worn dental prostheses.By: Ballestri, M; Baraldi, A; Gatti, AM; et al. GASTROENTEROLOGY Volume: 121 Issue: 5 Pages: 1234-1238 Published: NOV 2001)she defined the X-ray microprobe "radiograph microprobe" May be due to the scarce knowlege of the EDS mechanism, that imply an elemental analysis, the error usual in almost all the articles by Gatti is to state that what she find in the sample are non-biodegradable metals (The particles detected showed to contain highly-reactive, non-biocompatible and non-biodegradable metals. It is not specified to which of the white particles the spectra in fig. 1 refer. In my opinion, from the SEM images of the same figure, i.e. at such magnitude, it is almost impossible to measure the particles dimension. Further, even if the total surface occupied by the sample on the acetate filter is not declared, in my opinion, it is anyway almost impossible to count all the particles in a reasonable time; as an example the SEM image show an area of about 50x50 micrometers; how many images they have to acquire to cover a 10x10 mm area? The PCA is at all not explained, and not correctly graphicated. First of all the graph of the two Principal Components must be isometric and squared; the graph of the Loadings lacks and, looking at the data reported in tables and hystogram, it is unclear what they are; the first two components account for less than 47% of the total variance and the graph of the % variance as a function of the components lacks

On 2017 Feb 01, Davide Radice commented:

Visani G et al, compared the occurrence of nanoparticles and aggregates in the peripheral blood samples of AML patients and healthy subjects in a matched case-control study and based on their findings they argue that nanoparticles could lead to AML (that is: nanoparticles could be a risk factor for AML). However I found a number of issues regarding both the design and the statistical analysis. Here in brief the most relevant ones:

1. apart from the few number of subjects included they say that AML patients were matched with healthy subjects, however they do not specify which confounders they matched and controlled for (Table 1 only describes the characteristics of the AML patients but not those of the matched healthy subjects)

2. the primary analysis compares the average particles and aggregates counts, element by element (Table 4) through a series of two-sample t-tests: it should be kept in mind that

   a) the t-test is not suitable for count data because counts violate the underlying assumptions, thus any inference based on the significance of the t-test must be considered as possibly wrong

   b) despite they call each t-test as ‘independent’ because they consider controls as independent of cases, the individual tests are not independent of one another due to the fact that they all were conducted on the same sample of subjects. This is a well-known additional issue called ‘the multiple comparison problem’ they would have to take into account even if they used a more appropriate non-parametric alternative to the t-test. For example taking the 19 raw p-values in Table 4 it can be shown that after adjusting for the multiplicity using the Holm method [1], the only statistically significant comparisons are those regarding the average counts for Al (p = 0.019) and Ca (p = 0.019).

Moreover a statistically significant difference for the average of the counts between AML and healthy subjects it does not imply that aggregates and particles can be considered as possible risk factors, as can not be inferred in general as a risk factor any other variable just on the basis of the significance of the difference between two means (think for a while to a significant difference between the average shoes size, observed by chance). To conclude correctly about a risk you don’t compare two means, you must estimate and test the risks. The authors they should have properly analyze their data using a conditional logistic regression model [2]

Taking the data in Table 2 and assuming that each sample in a row shows the counts of a properly matched case-control for the unspecified confounders, I ran the appropriate statistical analysis (SAS 9.3) as described above. Taking the controls as reference the (multivariable) conditional logistic regression analysis clearly shows that neither aggregates nor particles are significant risk factors for AML. Here the results:

• particles : OR = 1.19 (95% CI: 0.83,1.69) p = 0.34
• aggregates: OR = 0.58 (95% CI: 0.09,3.83) p = 0.57

Legenda: OR = Odds Ratio, CI = Confidence Interval

[1] Holm, S. (1979). "A simple sequentially rejective multiple test procedure". Scandinavian Journal of Statistics. 6 (2): 65–70

[2] Breslow, N. E., et al. (1978). "Estimation of multiple relative risk functions in matched case-control studies.". American Journal of Epidemiology. 108 (4): 299–307

On 2016 Nov 16, Joe G. Gitchell commented:

Readers may find my post and the subsequent discussion of interest at this link (which includes relevant disclosures): http://conscienhealth.org/2016/11/finding-a-way-for-healthier-generations/

On 2016 Sep 29, Andrew Brown commented:

The ability to evaluate and compare this study is limited by missing methodoligical details about the outcome phenotype: various measurements of obesity and adiposity. Please excuse me if I missed the details somehow, and I hope the authors will consider adding details to better help the scientific community evaluate the authors' contribution to microbiome-obesity research.

The authors state that they evaluated three measures of abdominal adiposity, inlcuding subcutaneous fat mass (SFM). SFM is not a measurement of abdominal adiposity unless restricted to the abdomen. The methods do not make such a distinction, and instead only indicate the data were collected from DXA. The authors cite two articles in the methods with respect to adiposity phenotypes; neither describe how SFM was defined.

The authors state, "Visceral fat mass was calculated from one cross section of the whole body at L4–L5, the typical location of a CT slice;" no reference is provided to defend the reliability or appropriateness of such a method. For instance, some methods have used different lumbar positions and some people insist that DXA is inappropriate for visceral adipose tissue estimation.

The authors also dichotomize 'high' and 'low' phenotypes of adiposity measurements without explanation in Figure 2. The methods of dichotomizing also are not clear: "For each phenotype, individuals who were more than 1.5 standard deviations from the mean of the phenotype were assigned to high and low phenotype groups respectively." Does this mean those less than 1.5 SD below the mean were 'low' and those more than 1.5 SD were 'high'? This would eliminate a large portion of the sample (+/- 1.5 SD removed from the middle of the distribution would leave <15% of the sample if it was normally distributed). If, on the other hand, it was dichotomized on a single point value 1.5 SD above the mean (for instance), this has severe limitations because such cutoffs can be slid along the continuum to provide very different results. Thus, it is typically best to provide a theoretical basis for classification or to have a confirmation set (e.g., see Ivanescu AE, 2016).

It is also unclear if these dichotomized values were used throughout the rest of the manuscript (e.g., they appear to be in figure 4). This impairs the reader's ability to evaluate results and compare to new or old findings.

On 2016 Oct 24, Sean Ekins commented:

here are infographics http://www.slideshare.net/ekinssean/tb-infographic-euro-funding http://www.slideshare.net/ekinssean/tb-infographic

On 2016 Dec 05, Donald Forsdyke commented:

SELECTIVE PRESSURE TO CONSERVE VIRUS SPECIES IDENTITY

The authors correctly note that "the most obvious parameter associated with G + C content is the strength of molecular hybridization of polynucleotide duplexes" (1). Such hybridization controls recombination, which is favored when there is close sequence resemblance between different co-infecting viruses ("complete alignment conserved"), and is impeded when there is less sequence resemblance ("complete alignment variable"). The latter anti-recombination activity can be considered in relation to speciation mechanisms that initiate and retain taxonomic differentiations. As recently noted by Meyer et al., allied species of "viruses that infect the same [host] species and cell types are thought to have evolved mechanisms to limit recombination." Without such limitations the genomes would blend and co-infectants would lose their independence as distinct viral species. Mechanisms overcoming this selective disadvantage include "divergences in nucleotide composition and RNA structure that are analogous to pre-zygotic barriers in plants and animals" (2).

Thus, a nucleic acid region may be "conserved," not only because it encodes a protein (i.e. there is "protein pressure" on the sequence), but because it has a specific nucleotide composition (e.g. "GC-pressure"). While protein pressure mainly affects the first and second codon positions, GC-pressure can affect all codon positions. Indeed, at first and second codon positions there may be conflict between pressures, especially when protein pressure is high (i.e. in regions where amino acid conservation is high); then GC-pressure is constrained to vary only at the more flexible third codon position. In contrast, when protein pressure is low (i.e. in regions where amino acid conservation is low), then GC-pressure has greater freedom to affect all codon positions.

If, to avoid recombination, there is selective pressure on one branch of a diverging line to decrease its GC%, then it would be predicted that "the GC% of nucleotides encoding conserved amino acid (AA) residues" would be "consistently higher than that of nucleotides encoding variable AAs," where the pressure to decrease GC% has fuller rein to encompass all three codon positions (1). Conversely, it would be predicted that when there is pressure on a diverging line to increase GC%, then it would be predicted that the GC% corresponding to conserved codons would be consistently lower than that of non-conserved codons (e.g. Ebolavirus).

For flavivirus "the mean G% of the core conserved AA residues is higher (35%) than that of the variable AA residues (28%), but the mean G3% of the core conserved AA residues (28%) is similar to that of the variable AA residues (29%)" (1). While consistent with the above views, there is need for information on C3% and relative frequencies of synonymous codons (e.g. the two cysteine codons correspond either to low or high GC%). More details of selective anti-recombination pressures are presented elsewhere (3, 4). Similar considerations may apply to codon biases and GC% among mycobacteriophages (5).

1.Klitting R, Gould EA & de Lamballerie X (2016) G + C content differs in conserved and variable amino acid residues of flaviviruses and other evolutionary groups. Infection, Genetics and Evolution 45: 332-340.Klitting R, 2016

2.Meyer JR, Dobias DT, Medina SJ, Servilio L, Gupta A, Lenski RE (2016) Ecological speciation of bacteriophage lambda in allopatry and sympatry. Science 354: 1301-1304. Meyer JR, 2016

3.Forsdyke (2014) Implications of HIV RNA structure for recombination, speciation, and the neutralism-selectionism controversy. Microbes & Infect16:96-103. Forsdyke DR, 2014

4.Forsdyke DR (2016) Evolutionary Bioinformatics, 3rd edition. Springer, New York.

5.Esposito LA, Gupta S, Streiter F, Prasad A, Dennehy JJ (2016). Evolutionary interpretations of mycobacteriophage biodiversity and host-range through the analysis of codon usage bias. Microbiol Genomics 2(10), doi: 10.1099/mgen.0.000079. See arXiv preprint

On 2017 Jun 30, Seán Turner commented:

Traore et al. (2015) [PMID 27257486] and Vicino et al. (2016) [PMID 27660714] both claim DSM 29078 to be the type strain of Bacillus andreraoultii and Murdochiella massiliensis, respectively. DSM 29078 is not listed in the DSMZ online catalog of strains as of 30 June 2017.

On 2017 Feb 03, Robert Eibl commented:

Great work!

On 2016 Dec 08, Ole Jakob Storebø commented:

Spin and double spin ̶ the Letter to the Editor by Romanos et al. (2016) is indeed spinning.

Response to “Check and Double Check ̶ the Cochrane review by Storebø et al. is indeed flawed” 

(This letter was rejected by the editors in Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie).

Romanos et al. 2016 (Romanos M, 2016) continue to publish disagreements with the findings of our Cochrane systematic review (Storebø OJ, 2015) that do not have any meaningful effect on our estimate of effect size regarding methylphenidate for children and adolescents with ADHD. Our main point is that due to the very low quality of all the evidence one cannot state anything for sure about the true magnitude of the effect.

It is correct that a post-hoc exclusion of the four trials with co-interventions in both MPH and control groups and the one trial of preschool children will change the effect size from 0.77 to 0.89. We have responded several times to this group of authors (Storebø OJ, 2015, Storebø OJ, 2016, Storebø OJ, 2016, Storebø OJ, 2016, OJ Storebø et al, 2016: doi:10.1136/eb-2016-102499) regarding these trials which were included in keeping with our protocol which was published a priori (Storebø OJ, 2015).

We agree that there might be an effect of both Clonidine and behavioral therapy. However, the effects are balanced by their use as add-on therapies in both arms of the trials i.e. the methylphenidate and no-methylphenidate arms. Such an analysis would be a post hoc decision made purely to increase the effect size and would be in conflict with our reviewed protocol (Storebø OJ, 2015).

There is no evidence for providing a valid cut off score for the effect size of the standardized minimal clinical difference (SMD) that can be used by clinicians. When reporting a SMD one of the challenges facing researchers is to determine the significance of any differences observed and communicate this to clinicians who will apply the results of the systematic review to clinical practice.

The use of a Minimal Clinical Relevant Difference (MIREDIF) is a valid way to express the minimum clinically important improvement considered worthwhile by clinicians and patients (Copay AG, 2007). The variability of MIREDIF is also important, which is why we reported the 95% confidence intervals of the transformed mean value in our review. Even with a difference in means below the MIREDIF, a proportion of the patients will have a value above the MIREDIF. Similarly, a proportion of the patients will have a value below the MIREDIF.

The use of end-of-period data in cross-over trials is problematic due to the risk for “carry- over effect” (Cox DJ, 2008) and “unit of analysis errors” (http://www.cochrane-handbook.org.). In addition, we have tested for the risk of “carry-over effect”, by comparing trials with first period data to trials with end-of-period data in a subgroup analysis. This showed no significant subgroup difference, but this analysis has sparse data and one can therefore not rule out this risk. Even with no statistical difference in our subgroup analysis comparing parallel group trials to end-of-period data in cross-over trials, there was high heterogeneity and this could mean that the risk of “unit of analysis error” and “carry-over effect” was in fact real.

We have continued to argue that the well known adverse events of methylphenidate, such as the loss of appetite and disturbed sleep, can be detected by teachers. We highlighted this in our review (Storebø OJ, 2015) and answered this point in several replies to these authors (Storebø OJ, 2015, Storebø OJ, 2016, Storebø OJ, 2016, Storebø OJ, 2016, OJ Storebø et al, 2016: doi:10.1136/eb-2016-102499). It is not about controlling the amount of food children eat in the schoolyard or assessing their sleep quality at night. The well known adverse events of “loss of appetite” and “disturbed sleep” are easily observable by teachers as uneaten food left on lunch plates, yawning, general tiredness and even weight loss.

There is considerable evidence that trials sponsored by industry overestimate benefits and underestimate harms (Lundh A, 2012, other citations). We did receive a table for the Coghill 2013 trial from the authors. We did, however, not ask for information about funding as it was clearly stated in Coghill 2013 that this trial was funded by Shire Development LLC (Coghill D, 2013).

It is true that some participants in the MTA study (10%) allocated to the methylphenidate treatment were titrated to dextroamphetamine (Anonymous, 1999). We wanted to conduct a reanalysis of the data excluding the participants who did not receive methylphenidate. We contacted Dr. Swanson and he proved several helpful comments. He also enclosed published articles, but we did not receive additional data, in part because of the time frame of our review (Storebø OJ, 2015). Sensitivity analysis excluding the MTA study does not significantly change the effect estimate.

We have seriously considered the persistent, repeated criticisms by Ramonas et al. published in a number of different journals, however, none of these have provided evidence which justify changing our conclusions about the effects of MPH and the very low quality of evidence of methylphenidate trials. We had no preconceptions of the findings of this review and followed the published protocol, therefore any proposed manipulations of the data proposed by this group of authors would be in contradiction to the accepted methods of high quality meta-analyses. As it is unlikely that any further criticisms from these authors will change and we feel we have repeatedly responded clearly to each of these criticism, we propose to agree to disagree.

Ole Jakob Storebø, Psychiatric Research Unit, Psychiatric Department, Region Zealand, Denmark

Morris Zwi, Islington CAMHS, Whittington Health, London, UK

Carlos Renato Moreira-Maia, Federal University of Rio Grande do Sul, Porto Alegre, Brazil

Camilla Groth, Pediatric Department, Herlev University Hospital, Herlev, Denmark

Donna Gillies, Western Sydney Local Health District; Mental Health, Parramatta, Australia

Erik Simonsen, Psychiatric Research Unit, Psychiatric Department, Region Zealand, Denmark

Christian Gluud, Copenhagen Trial Unit, Centre for Clinical Intervention Research, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark

On 2017 Feb 04, Misha Koksharov commented:

It's quite interesting that this type of cyclization doesn't prevent the necessary C-domain rotation and luciferase remains fully active.

Fig. 8 needs some corrections: 1) The pH values are swapped in the figure caption [(A) pH 5.5 and (B) pH 7.8]. 2) Probably, there is some problem with pH adjustment for pH 5.5. P. pyralis luciferase should give nearly monomodal red spectrum at pH 5.5, maybe with a faint green shoulder (Branchini BR, 2007, Oba Y, 2012, Riahi-Madvar A, 2009). Here it is only a faint red shoulder.

On 2016 Oct 22, Lydia Maniatis commented:

As is well-known, perceived color and lightness are contingent on the distribution of luminance/chromaticity in the proximal stimulus and on the corresponding structure of the percept, via the visual process and the principles instantiated therein.

This being the case, any conclusions regarding perceived color contrast that do not explicitly include structure in the discussion will not generalize but will be strictly ad hoc. The results of the present study apply to "plaids", and surely not all plaids, as a category, given the relevance of particular chromaticity values and distributions. The title of the paper, however, implies generality.

The authors also inappropriately leave structure out of the conversation when they speculate that:

"Superimposed luminance and color contrast without co-alignment commonly occurs in natural scenes when shadows or shading fall on a colored surface, whereas co-aligned color and luminance borders are indicative of object and material boundaries, suggesting these two situations activate different color-luminance interactions. "

As descriptions the terms "superimposed luminance and color contrast without co-alignment" and "co-aligned color and luminance borders" are not specific enough, being structure-blind, to assure whether and where "shadows" and "object boundaries" will result in the percept. An easy example, always to hand are the amodally-completed boundaries (which occur in addition to the subjective contours) in the Kanizsa triangle, as well as the appearance of overlap within figures without a luminance step, as well as situations in which the color spectrum or intensity range is shifted and which produce the impression of colored illumination or shading. Whether two areas are interpreted as overlapping ("superimposed" is a description of the percept not the proximal stimulus or the stimulus presented on a screen) depends on relative surface properties and their shapes, not simple "alignments."

So to refer to two different "situations" on the basis of the result - the percept - and to assume different mechanisms ("different color-luminance interactions), without adequately specifying a priori how these two situations are differentiated with respect to the stimulus structure, is putting the cart before the horse.

On 2016 Oct 11, Alem Matthees commented:

References for the above comment

1) Hawkes N. Freedom of information: can researchers still promise control of participants' data? BMJ. 2016 Sep 21;354:i5053. doi: 10.1136/bmj.i5053. PMID: 27654128. http://www.bmj.com/content/354/bmj.i5053

2) White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, Baber HL, Burgess M, Clark LV, Cox DL, Bavinton J, Angus BJ, Murphy G, Murphy M, O'Dowd H, Wilks D, McCrone P, Chalder T, Sharpe M; PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet. 2011 Mar 5;377(9768):823-36. doi: 10.1016/S0140-6736(11)60096-2. Epub 2011 Feb 18. PMID: 21334061. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3065633/

3) Confidentiality: NHS Code of Practice. November 2003. https://www.gov.uk/government/publications/confidentiality-nhs-code-of-practice

4) General Medical Council (2009). Confidentiality guidance: Research and other secondary uses. http://www.gmc-uk.org/guidance/ethical_guidance/confidentiality_40_50_research_and_secondary_issues.asp

5) Queen Mary University of London. PACE trial protocol: Final version 5.2, 09.03.2006. A1.6-A1.7 [Full Trial Consent Forms] https://www.whatdotheyknow.com/request/203455/response/508208/attach/3/Consent forms.pdf

6) Appeal to the First-tier Tribunal (Information Rights), case number EA/2015/0269: http://informationrights.decisions.tribunals.gov.uk/DBFiles/Decision/i1854/Queen Mary University of London EA-2015-0269 (12-8-16).PDF

7) Tracking switched outcomes in clinical trials. http://compare-trials.org/

8) White PD, Chalder T, Sharpe M, Johnson T, Goldsmith K. PACE trial authors' reply to letter by Kindlon. BMJ. 2013 Oct 15;347:f5963. doi: 10.1136/bmj.f5963. PMID: 24129374. http://www.bmj.com/content/347/bmj.f5963

9) Goldsmith KA, White PD, Chalder T, Johnson AL, Sharpe M. The PACE trial: analysis of primary outcomes using composite measures of improvement. Queen Mary University of London. 8 September 2016. http://www.wolfson.qmul.ac.uk/images/pdfs/pace/PACE_published_protocol_based_analysis_final_8th_Sept_2016.pdf

10) Wikipedia. Multiple comparisons problem. Accessed 02 October 2016. https://en.wikipedia.org/wiki/Multiple_comparisons_problem

11) Matthees A, Kindlon T, Maryhew C, Stark P, Levin B. A preliminary analysis of ‘recovery’ from chronic fatigue syndrome in the PACE trial using individual participant data. Virology Blog. 21 September 2016. http://www.virology.ws/wp-content/uploads/2016/09/preliminary-analysis.pdf

12) Tuller D. Trial By Error, Continued: Questions for Dr. White and his PACE Colleagues. Virology Blog. 4 January 2016. http://www.virology.ws/2016/01/04/trial-by-error-continued-questions-for-dr-white-and-his-pace-colleagues/

13) Walwyn R, Potts L, McCrone P, Johnson AL, DeCesare JC, Baber H, Goldsmith K, Sharpe M, Chalder T, White PD. A randomised trial of adaptive pacing therapy, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome (PACE): statistical analysis plan. Trials. 2013 Nov 13;14:386. doi: 10.1186/1745-6215-14-386. PMID: 24225069. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4226009/

14) White PD, Goldsmith K, Johnson AL, Chalder T, Sharpe M. Recovery from chronic fatigue syndrome after treatments given in the PACE trial. Psychol Med. 2013 Oct;43(10):2227-35. doi: 10.1017/S0033291713000020. PMID: 23363640. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3776285/

15) Beth Smith ME, Nelson HD, Haney E, et al. Diagnosis and Treatment of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Rockville (MD): Agency for Healthcare Research and Quality (US); 2014 Dec. (Evidence Reports/Technology Assessments, No. 219.) July 2016 Addendum. Available from: http://www.ncbi.nlm.nih.gov/books/NBK379582/

16) Matthees A. Treatment of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Ann Intern Med. 2015 Dec1;163(11):886-7. doi: 10.7326/L15-5173. PMID: 26618293. http://www.ncbi.nlm.nih.gov/pubmed/26618293

17) Kennedy A. Authors of our own misfortune?: The problems with psychogenic explanations for physical illnesses. CreateSpace Independent Publishing Platform. 4 September 2012. ISBN-13: 978-1479253951. https://www.amazon.com/Authors-our-own-misfortune-explanations/dp/1479253952

18) Sharpe M, Goldsmith KA, Johnson AL, Chalder T, Walker J, White PD. Rehabilitative treatments for chronic fatigue syndrome: long-term follow-up from the PACE trial. Lancet Psychiatry. 2015 Dec;2(12):1067-74. doi: 10.1016/S2215-0366(15)00317-X. Epub 2015 Oct 28. PMID: 26521770. https://www.ncbi.nlm.nih.gov/pubmed/26521770

19) Higgins PTJ, Altman DG, Sterne JAC; on behalf of the Cochrane Statistical Methods Group and the Cochrane Bias Methods Group. Chapter 8: Assessing risk of bias in included studies. Version 5.1.0 [updated March 2011]. http://handbook.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm

20) Schulz KF, Grimes DA. Blinding in randomised trials: hiding who got what. Lancet. 2002 Feb 23;359(9307):696-700. PMID: 11879884. http://www.who.int/rhl/LANCET_696-700.pdf

21) Boot WR, Simons DJ, Stothart C, Stutts C. The Pervasive Problem With Placebos in Psychology: Why Active Control Groups Are Not Sufficient to Rule Out Placebo Effects. Perspect Psychol Sci. 2013 Jul;8(4):445-54. Doi: 10.1177/1745691613491271. PMID: 26173122. http://pps.sagepub.com/content/8/4/445.long

22) Button KS, Munafò MR. Addressing risk of bias in trials of cognitive behavioral therapy. Shanghai Arch Psychiatry. 2015 Jun 25;27(3):144-8. doi: 10.11919/j.issn.1002-0829.215042. PMID: 26300596. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4526826

23) Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336 (7644): 601–5. DOI:10.1136/bmj.39465.451748.AD. PMID 18316340. PMC 2267990. http://www.bmj.com/cgi/content/full/336/7644/601

24) Kindlon TP. Objective measures found a lack of improvement for CBT & GET in the PACE Trial: subjective improvements may simply represent response biases or placebo effects in this non-blinded trial. BMJ Rapid Response. 18 January 2015. http://www.bmj.com/content/350/bmj.h227/rr-10

25) Knoop H, Wiborg J. What makes a difference in chronic fatigue syndrome? Lancet Psychiatry. 2015 Feb;2(2):113-4. doi: 10.1016/S2215-0366(14)00145-X. Epub 2015 Jan 28. PMID: 26359736. https://www.ncbi.nlm.nih.gov/pubmed/26359736

26) johnthejack (Peters J). Using public money to keep publicly funded data from the public. 29 June 2016. https://johnthejack.com/2016/06/29/using-public-money-to-keep-publicly-funded-data-from-the-public/

27) Coyne JC. Further insights into war against data sharing: Science Media Centre’s letter writing campaign to UK Parliament. 31 January 2016. https://jcoynester.wordpress.com/2016/01/31/further-insights-into-the-war-against-data-sharing-the-science-media-centres-letter-writing-campaign-to-uk-parliament/

On 2016 Oct 11, Alem Matthees commented:

On 4 October 2016 I submitted a BMJ Rapid Response to this article but one week later it has not been posted on the BMJ website ( http://www.bmj.com/content/354/bmj.i5053/rapid-responses ). I am not sure whether this delay is normal or if it has been rejected. So I will post it on PubMed Commons. The following is as previously submitted, except for the addition of one word to help clarify a sentence, and the addition of a PMID for reference 22 (and removing a stray character). I also had to move the references to a separate PubMed Commons comment below this one:

The PACE trial investigators never had total control over the data to begin with

Thank you for covering this issue. Some comments:

a) This article states that it was “not possible” to contact me[1]. However, my email address shows up in the first page of results with a Google search for Alem Matthees.

b) Regarding the modification of consent forms for future trials to address FOIA data releases. The FOIA was implemented in January 2005, before PACE trial participants were recruited[2]; under the legislation, trial data that is unlikely to identify participants is not personal data, and it was always QMUL’s responsibility to be aware that trial data is within scope of the FOIA, but they failed to inform participants of this possibility. Similarly, confidentiality guidelines from the NHS[3] and GMC[4] state that consent is not necessary to release de-identified data. The trial consent form promised that identities will be protected[5], and they have been. The Information Commissioner and Information Tribunal considered and rejected the assertions that FOIA data releases would significantly affect recruitment in future studies[6].

c) The lesson here is not about ‘controlling’ data, it is that if data is not analysed or published in a fair and transparent way, people will seek to acquire and re-analyse it, particularly when debatable claims were made that affect the lives of millions of patients. The major deviations from the published trial protocol, the recovery criteria in particular, is what motivated me. Outcome switching is recognised as a major problem in the research community[7].

While it was important to find out the protocol-specified primary outcomes that were abandoned, the changes to the recovery criteria (a secondary analysis) were the most problematic. I sought the data after QMUL refused to release the protocol-specified outcomes for improvement and recovery. It is misleading to promote ‘recovery’ rates of 22% when based on indefensible criteria, such as thresholds of ‘normal’ fatigue and physical function that overlap with trial eligibility criteria for severe disabling fatigue, and where one-third still met Oxford CFS criteria.

d) White et al. previously downplayed the changes to the primary outcomes as the primary measures “were the same as those described in the protocol”[8]. Now that the results for the protocol-specified primary outcomes are known and people are comparing them with the post-hoc equivalents, Professor White is arguing that “They’re not comparing like with like […] They are comparing one measure with a completely different one—it’s apples and pears”.[1]

Professor White also stated that going back to the protocol makes no difference, as adjunctive CBT and GET are still statistically significantly better than specialist medical care alone[1]. However, statistical significance is not the same as clinical significance, and going back to the protocol decreases the response rates in the CBT and GET groups from approximately 60% down to 20% (compared to 45% down to 10% for SMC alone)[9].

While it may be argued that the above does not change the conclusion that adjunctive CBT and GET are superior to SMC alone, the trial investigators conducted many analyses without correcting for multiple comparisons[10]; based on a quick look at the p values, some of the differences reported may not be statistically significant when using a more conservative approach.

Moreover, the data has been re-analysed and going back to the protocol not only decreased the 'recovery' rates from 7-22% down to to 2-7%, but the differences between adjunctive therapy groups and SMC alone are not significant[11]. There appears to be a consistent pattern of outcome switching and major changes to thresholds that inflate the results by several times over.

e) The article states that White et al. “had answered critics who have made legitimate scientific points”. However, there are numerous legitimate questions or problems that are unaddressed[12].

f) The article states that out of 37 FOIA requests made, “many” were rejected as vexatious. But only 3 or so have been rejected under S.14 (e.g. see whatdotheyknow.com), the first one was in relation to details about the timing and nature of the changes to the protocol, the other two or so by others, relating to trial data. Asking for trial data or for details about methodology is not harassment.

g) It remains unclear whether all the changes to the trial protocol were made and independently approved before analysing data. Statements about this issue appear to relate to the 2011 Lancet paper only, but there is no mention of change to the recovery criteria in the statistical analysis plan that was finalised shortly before the unmasking of data[13]. The ‘normal range’ is described in the 2011 Lancet paper as a post-hoc analysis[2], and this ‘normal range’ then formed part of the revised recovery criteria published in Psychological Medicine in 2013[14] without any mention of approval. I urge the trial investigators to clarify once and for all whether the changes to the recovery criteria were made after the unmasking of any trial data and whether these were independently approved.

h) Patients want to get better but many are simply not impressed with the methodology or results of PACE: 80% of candidates definitely or provisionally diagnosed with CFS were excluded from the trial[2]. The CFS and ME case criteria used were problematic[15-17]. Only a small minority of broadly defined CFS patients reported benefit from CBT or GET (around 10-15% over SMC). That benefit was modest and transient, with no significant advantages at 2.5 year follow-up[18].

Subjective self-reports are important, but modest improvements are difficult to separate from a placebo response and other reporting biases when a trial is non-blinded and tests therapies that aim to change patients’ perceptions about their illness[19-23]. This issue becomes more relevant given that there was a complete absence of meaningful improvements to multiple objective outcomes[24] (the small improvement in walking distance for the GET group has been attributed by CBT/GET proponents to participants pushing themselves harder on the test rather than being fitter[25]).

i) QMUL spent £245,745 on legal fees trying to prevent release of the requested data[26], and were also part of a failed lobbying attempt to be removed from the FOIA[27].

References

[continued below...]

On 2016 Sep 26, Peter Hajek commented:

The title asserts that smokers who switch to vaping start to drink more. The paper however shows no such thing. It just reports that ex-smokers who vape drink more than ex-smokers who do not vape. Heavier smokers are more likely to seek nicotine maintenance and are also heavier drinkers and the difference almost certainly predated quitting. The title does not reflect the study findings and misleads casual readers.

On 2016 Sep 30, Wasim Maziak commented:

None

On 2016 Sep 26, Peter Hajek commented:

The letter has a misleading title, it identifies no 'unsubstantiated claims'. It just says that some ex-smokers would quit anyway and that surveys have a margin of error. It raises no material issues that would suggest that the article's finding of a huge number of smokers who claim to have stopped smoking with the help of e-cigarettes is inaccurate.

On 2016 Oct 27, James Yeh commented:

Editor's Comment Obesity and Management of Weight Loss — Polling Results

James Yeh, M.D., M.P.H., and Edward W. Campion, M.D.

Obesity is increasingly prevalent worldwide, and about 40% of Americans meet the diagnostic criteria for obesity.[1] The goal of weight loss is to reduce the mortality and morbidity risks associated with obesity. Patients with a body-mass index (BMI) in the range that defines obesity (>30) have a risk of death that is more than twice that of persons with a normal BMI.[2] Obesity is also associated with increased risks of cardiovascular disease, diabetes, and several cancers. A recent study suggests that being overweight or obese during adolescence is strongly associated with increased cardiovascular mortality in adulthood.[3] Studies suggest that even a 5% weight loss may reduce the complications associated with obesity.[4]

In September 2016, we presented the case of Ms. Chatham, a 29-year-old woman with class I obesity (BMI, 32) who leads a fairly sedentary lifestyle, with frequent reliance on takeout foods and with infrequent physical activity.[5] Readers were invited to vote on whether to recommend initiating treatment with one of the FDA-approved drugs for weight loss along with lifestyle modifications or to recommend only nonpharmacologic therapies and maximizing lifestyle changes. The patient has no coexisting medical conditions, but her blood pressure is slightly elevated (144/81 mm Hg). In the past, Ms. Chatham has tried to lose weight using various diets, each time losing 10 to 15 lb (4.5 to 6.8 kg), but she has never been able to successfully maintain weight loss.

Over 85,000 readers viewed the Clinical Decisions vignette during the polling period, and 905 readers from 91 countries voted in the informal poll. The largest group of respondents (366) was from the United States or Canada, representing nearly 40% of the votes. A large majority of the readers (80%) voted against prescribing one of the FDA-approved medications for weight loss and instead recommended maximizing lifestyle modification and nonpharmacologic therapies first.

A substantial proportion of the 64 Journal readers who submitted comments expressed concern about the absence of efficacy data on long-term follow-up and about the side effects associated with current FDA-approved medications for weight loss. Some suggested that simply treating obesity with a prescription medication is shortsighted and that it is important to uncover patients’ motivations for existing lifestyle choices and for weight loss. The commenters emphasized the need for a multifaceted approach to obesity management that includes nutritional and psychological support, as well as stress management, with the goal of long-lasting improvement in exercise and eating habits that will lead to weight reduction and maintenance of a healthier weight.

Some commenters, noting the difficulty of lifestyle changes, felt that pharmacotherapy can be a complementary and reasonable part of a multidisciplinary treatment plan. Some wrote that obesity should be managed as a chronic disease is managed and that an inability to lose weight should not be seen as a disciplinary issue, especially given the importance of genetic and physiological factors. These commenters argued that the use of pharmacotherapy as part of the treatment plan to achieve weight loss should not be stigmatized.

Overall, the results of this informal Clinical Decisions poll indicate that a majority of the respondents think physicians should not initially recommend the use of an FDA-approved drug as part of a weight-loss strategy, at least not for a patient such as Ms. Chatham, and that many respondents were troubled by the current uncertainties about the long-term efficacy and safety of weight-loss drugs.

REFERENCES 1. Flegal KM, Kruzon-Moran D, Carroll MD, Fryar CD, Ogden CL. Trends in obesity among adults in the United States, 2005 to 2014. JAMA 2016;315:2284-91. 2. Global BMI Mortality Collaboration. Body-mass index and all-cause mortality: individual-participant-data meta-analysis of 239 prospective studies in four continents. Lancet 2016;388:776-86. 3. Twig G, Yaniv G, Levine H, et al. Body-mass index in 2.3 million adolescents and cardiovascular death in adulthood. N Engl J Med 2016;374:2430-40. 4. Kushner RF, Ryan DH. Assessment and lifestyle management of patients with obesity: clinical recommendations from systematic reviews. JAMA 2014;312:943-52. 5. Yeh JS, Kushner RF, Schiff GD. Obesity and management of weight loss. N Engl J Med 2016;375;1187-9.

On 2016 Oct 11, Richard Kellermayer commented:

Great work!

It would have been nice, however, to see our work on the mucosal mycobiome in pediatric Crohn disease patients discussed and referenced:

Microbiota separation and C-reactive protein elevation in treatment-naïve pediatric granulomatous Crohn disease. Kellermayer R, Mir SA, Nagy-Szakal D, Cox SB, Dowd SE, Kaplan JL, Sun Y, Reddy S, Bronsky J, Winter HS. J Pediatr Gastroenterol Nutr. 2012 Sep;55(3):243-50. doi: 10.1097/MPG.0b013e3182617c16. PMID: 22699834

On 2016 Oct 03, Atanas G. Atanasov commented:

"Many "rules" for writing good science abstracts associate with fewer citations":

https://twitter.com/mattjhodgkinson/status/594865422840762368 ...and... http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004205

...another study pointing in the same direction as our work.

On 2016 Sep 21, Murat Büyükşekerci commented:

In my opinion there is problem with title of this article. Since the term "mediator" refers to intracellular proteins that enhance and activate the functions of other proteins. Thiol/disulphide homeostasis is a term used to describe the redox state of mileu and could not be defined as a novel mediator. Thanks for regarding.

On 2016 Oct 16, Thomas Langer commented:

Loss of m-AAA proteases increases mitochondrial Ca2+ influx at low cytosolic [Ca2+]

We demonstrate in our paper that the m-AAA protease (AFG3L2/SPG7) degrades EMRE, an essential subunit of the mitochondrial Ca2+ uniporter MCU. Loss or decrease of m-AAA protease activity, as observed in SCA28, impairs the assembly of MCU with the gatekeeper subunits MICU1/2 and results in the formation of unregulated, open MCU. This causes an increased mitochondrial Ca2+ influx at low cytosolic [Ca2+] and renders neurons more susceptible to Ca2+ overload, opening of the mitochondrial permeability transition pore (MPTP) and cell death. Thus, we do not propose in our manuscript that the formation of deregulated MCU causes an increase in cytosolic [Ca2+], as suggested in the comment by Casari et al.. Our findings explain the striking observation by the Casari group that reduced Ca2+ influx into AFG3L2-deficient neurons (by pharmacological inhibition or genetic ablation of mGluR1) protects against neuronal death (Maltecca et al., 2015): lowered cytosolic [Ca2+] in these settings result in decreased mitochondrial Ca2+ influx via deregulated MCU lacking gatekeeper subunits in AFG3L2-deficient neurons, thus preventing mitochondrial Ca2+ overload. Of note, our findings are also in agreement with two recent studies in MICU1-deficient mice demonstrating that deregulated Ca2+ influx causes MPTP opening-induced cell death (Antony et al., Nat. Com., 2016) and ataxia by specifically affecting Purkinje cells (Liu et al., Cell Reports, 2016). Strikingly, reduced EMRE expression was found to suppress ataxia (Liu et al., Cell Reports, 2016).

Casari et al. have suggested that other (yet poorly understood) functions of the m-AAA protease lower the mitochondrial membrane potential (Maltecca et al., 2015) and impair mitochondrial morphology (Maltecca et al., 2012), resulting in decreased mitochondrial Ca2+ influx. Our results do not support a major role of disturbed mitochondrial morphology (Fig. 7), but we agree (and confirm in Fig. S6) that lowering the mitochondrial membrane potential decreases mitochondrial Ca2+ influx after histamine stimulation. We therefore have assessed mitochondrial Ca2+ influx upon mild increase of cytosolic [Ca2+] and observed an increased Ca2+ influx into m-AAA protease-deficient mitochondria (Fig. 6). The rationale of this protocol relies on the sigmoidal relationship between mitochondrial Ca2+ influx and extramitochondrial [Ca2+]. In resting conditions, mitochondrial Ca2+ accumulation is negligible when cytosolic [Ca2+] is below a threshold (~500 nM). Inhibition of SERCA leads to ER Ca2+ leaks, thus causing a slow and small increase of cytosolic [Ca2+]. In this experimental setup (low cytoplasmic [Ca2+]), mitochondrial Ca2+ influx is less hampered by a reduced mitochondrial membrane potential and indeed we observed an increased mitochondrial Ca2+ influx in AFG3L2-deficient mitochondria. We therefore suggest (and discuss in our manuscript) that m-AAA protease-deficient mitochondria show increased Ca2+ influx at resting [Ca2+] but decreased Ca2+ influx at high Ca2+ concentrations (due to the lowered membrane potential).

Casari et al. also raise doubts about the relative role of Ca2+ and mtROS for MPTP opening. We demonstrate a reduced Ca2+ retention capacity of AFG3L2-deficient mitochondria in vitro and in vivo, which correlates with the increased mitochondrial Ca2+ influx (observed upon SERCA inhibition) and the increased ROS levels in AFG3L2-deficient mitochondria. Increased mitochondrial Ca2+ influx under resting conditions is known to trigger MPTP opening (Antony et al., Nat. Com., 2016) and to cause increased mtROS production (Hoffman et al., Cell Reports, 2013; Mallilankaraman et al., Cell, 2012). Thus, both events are interdependent and their relative contribution to MPTP opening is difficult to dissect. We have not addressed this issue in the present manuscript and, by no means, exclude a contribution of mtROS to MPTP opening.

Together, our results provide compelling evidence that m-AAA protease deficiency causes the accumulation of MCU-EMRE complexes lacking gatekeeper subunits and impairs mitochondrial Ca2+ handling, sensitizing neurons for MPTP opening. The relative contribution of deregulated mitochondrial Ca2+ influx and lowered mitochondrial membrane potential for disease pathogenesis is currently difficult to assess and certainly warrants further studies in appropriate mouse models. However, we would like to point out that other mitochondrial diseases affecting respiration and the formation of the mitochondrial membrane potential do not show the striking vulnerability of Purkinje cells seen in SCA28. At the same time, MCU-dependent mitochondrial Ca2+ influx is a crucial determinant of excitotoxicity in neurons (Qui et al., Nat. Com., 2013). This study also demonstrates that synaptic activity transcriptionally suppresses MCU expression thereby counteracting mitochondrial Ca2+ overload at high cytosolic [Ca2+] and preventing induction of excitotoxicity. Our results thus open up the attractive possibility that increased Ca2+ influx under resting conditions and the accompanying mild stress increases progressively the vulnerability of Purkinje cells, causing late-onset neurodegeneration in SCA28 patients, which are only heterozygous for mutations in AFG3L2.

On 2016 Oct 13, Giorgio Casari commented:

Increased or decreased calcium influx?

In this elegant paper the authors propose that loss of m-AAA (i.e. the depletion of both SPG7 and AFG3L2) facilitates the formation of active MCU complexes through the increased availability of EMRE, thus (i) increasing calcium influx into mitochondria, (ii) triggering MPTP opening and (iii) causing the consequent increase of neuronal cytoplasmic calcium leading to neurodegeneration. We previously reported that loss or reduction of AFG3L2 causes (i) decreased mitochondrial potential and fission, thus (ii) decreased calcium entry and (iii) the consequent augmented neuronal cytoplasmic calcium leading to neurodegeneration. While the functional link of m-AAA with MAIP and MCU-EMRE represents a new milestone in the characterization of the roles this multifaceted protease complex, we would like to comment on the conclusions pertaining to the calcium dynamics. 1. In SPG7/AFG3L2 knock-down HeLa cells (Figure S6A) mitochondrial matrix calcium is dramatically reduced (approx. from 100 to 50 microM) following histamine stimulation, which triggers IP3-mediated calcium release from ER. This reduction is in complete agreement with the one we previously detected in Afg3l2 ko MEFs (Maltecca et al., 2012), and that we also confirmed in Afg3l2 knock-out primary Purkinje neurons (the cells that are primarily affected in SCA28) upon challenge with KCl (Maltecca et al., 2015). The decreased mitochondrial calcium uptake correlates with the 40% reduction of mitochondrial membrane potential in SPG7/AFG3L2 knock-down cells (Figure S6B), as expected since the mitochondrial potential is the major component of the driving force for calcium uptake by MCU. Accordingly, these data are in line with the decreased mitochondrial membrane potential observed in Afg3l2 knock-out Purkinje neurons (Maltecca et al., 2015). We think that this aspect is central, because the respiratory defect is the primary event associated to m-AAA deficiency and neurodegeneration. So, the data of König et al. agree with our own findings that mitochondrial matrix calcium is reduced after m-AAA depletion. 2. By a different protocol (SERCA pumps inhibition and ER calcium leakage; Figure 6 C-F), the authors detected a small increase of mitochondrial calcium concentration in SPG7/AFG3L2 knock-down HeLa cells (from approx. 3 to 6 microM). The huge difference in calcium concentration detected in the two experiments (100 to 50 microM in Figure S6A and 3 to 6 microM in Figure 6 C) possibly reflects the stimulated (histamine) vs. unstimulated (calcium leakage) conditions, this latter being more difficult to correlate to physiologic neuronal situation. 3. The authors show increased sensitivity to MPTP opening in the absence of m-AAA and they propose the consequent calcium release as the cause of calcium deregulation and neuronal cell death. ROS are strong sensitizers of MPTP to calcium and thus favor its opening. It is well known that m-AAA loss massively increases intramitochondrial ROS production. Thus, higher ROS levels, rather than high calcium concentrations, can be the trigger of MPTP opening. Taking all this into consideration, we think that mitochondrial depolarization (as shown in Figure S6B) and decreased mitochondrial calcium entry (Fig S6A), even in the presence of increased amount of MCU-EMRE complexes, may lead to inefficient mitochondrial calcium buffering and, finally, to cytoplasmic calcium deregulation. ROS dependent MPTP opening, which may occur irrespective of a low matrix calcium concentration, may additionally contribute to this final event.

Minor comment At page 7 we read: “Notably, these experiments likely underestimate the effect on mitochondrial Ca2+ influx observed upon loss of the m-AAA protease, since the loss of the m-AAA protease also decreases ΔΨ (i.e., the main force driving mitochondrial Ca2+ influx), as revealed by the significant impairment of mitochondrial Ca2+ influx triggered by histamine stimulation (Maltecca et al., 2015) (Figures S6A–S6E)”. The reference is not appropriate, since in Maltecca et al., 2015 the reduced mitochondrial calcium uptake has been demonstrated in Afg3l2 knock-out Purkinje neurons upon challenge with KCl and not with histamine. We used histamine stimulation, which triggers IP3-mediated calcium release from ER, in Afg3l2 ko MEF in a previous publication (Maltecca F, De Stefani D, Cassina L, Consolato F, Wasilewski M, Scorrano L, Rizzuto R, Casari G. Respiratory dysfunction by AFG3L2 deficiency causes decreased mitochondrial calcium uptake via organellar network fragmentation. Hum Mol Genet. 2012, 21:3858-70. doi: 10.1093/hmg/dds214).

On 2016 Nov 30, Gwinyai Masukume commented:

According to this 2016 Maternal Health Lancet Series paper, unlike for most African countries no data was available for the number of obstetricians and gynaecologists and midwives in South Africa precluding the calculation of the ratio of these practitioners per 1000 pregnancies. This deficit of South African data also applies to the Caesarean section rate global estimates from the World Health Organization published this year in another journal Betrán AP, 2016.

The apparent lack of data from South Africa suggests that it has ‘fallen off’ the international maternal health map. However, the Health Professions Council of South Africa Holmer H, 2015 and the South African Nursing Council have contemporary and historical data on the number of obstetricians and gynaecologists and midwives respectively. Caesarean delivery rates are available from the Health Systems Trust.

Because information from The Lancet and the World Health Organization has a global reach and influences key policy makers, this high level lack of visibility of pertinent South African maternal health data is concerning. Maternal health metrics are “essential to guide intervention research, set implementation priorities, and improve quality of care, particularly for women and babies most at risk” Koblinsky M, 2016.

Efforts to quickly address this lack of visibility are warranted.

On 2017 Feb 06, GARRET STUBER commented:

*This review was completed as part of a graduate level circuits and behavior course at UNC-Chapel Hill. The critique was written by students in the class and edited by the instructor, Garret Stuber.

Comments and critique

Written by Li et al., this paper investigated a class of oxytocin receptor interneurons (OxtrINs) on which the same group first characterized in 2014 [1]. OxtrINs are a subset of somatostatin positive interneurons in the medial prefrontal cortex (mPFC) that seem to be important for sociosexual behaviors in females, specifically during estrus and not diestrus. To complement their previous story, here the authors concluded that OxtrINs in males regulate anxiety-related behaviors through the release of corticotropin releasing hormone binding protein (Crhbp). While we agree that these neurons could be mediating sexually dimorphic behaviors, it is unclear how robust these differences really are.

We had some technical issues with this paper. First, it is unclear exactly how many mice were allotted to each experimental group, and it would have been useful to see individual data in each of the behavioral experiments, so that we can better understand some of the variability in the authors’ graphs. Even among different experiments, there were variable sizes of n (e.g. Fig. 5F-H, “n = 8-14 mice per group”). There was also no mention of how many cells per animal were tested for each brain slice experiment; instead, we received total numbers of cells tested per group. This paper did not include the complementary female data to Fig. 4F-G and Fig. 5A-B, the experiments pairing blue light with Crhr1 antagonist or Crhbp antagonist. We would have appreciated seeing this data adjacent to that for the males. In addition, there was no mentioned control for the optogenetic experiments. The authors only compared responses between light on and light off trials. Typically in optogenetic approaches, a set of control mice are also implanted with optic fibers and flashed with blue light in the absence of virus to test whether the light alone influences behavior. Incidentally, there is evidence that blue light influences blood flow, which may affect neuronal activity [2]. It was also unclear during the sociosexual behavioral testing whether the males were exposed to females in estrus or diestrus. In all, lack of detailed sample sizes and controls made it difficult to assess how prominent these sex differences were.

These issues aside, knocking out endogenous Oxtr in their targeted interneuron population was a key experiment, as it demonstrated that oxytocin signaling in OxtrINs is important in anxiety-related behaviors in males, but not in females regardless of the estrus stage. They did this using a floxed Oxtr mouse and deleted OxtR using a Cre-inducible virus, allowing for temporal and cell-type-specific control of this deletion, and subsequently measured the resulting phenotype using an elevated plus maze and open field task. The authors also validated that changes in exploration were not due to hyperactivity. We think these experiments are convincing.

TRAP profiling, which the same research group pioneered in 2014 [3], provided a set of genes enriched in OxtrINs. TRAP targets RNAs while they are translated into proteins, so we think their results here are particularly relevant. Moreover, the authors provided a list of genes enriched in sex-specific OxtrINs, a useful resource for those interested in gene expression differences in males and females. Once they identified Crhbp, an inhibitor of Crh, they hypothesized that OxtrINs were releasing Crhbp to modulate anxiogenic behaviors in males. The authors next measured Crh levels in the paraventricular nucleus of the hypothalamus and found that Crh levels are higher in females than males. They thus concluded Crh levels were driving sex differences associated with OxtrINs. We wonder whether Crh levels are also higher in the female mPFC, but we agree here too.

To demonstrate that Crhbp expressed by OxtrINs is important in modulating anxiety-like behaviors in males, the authors targeted Crhbp mRNA using Cre-inducible viral delivery of an shRNA construct and subsequently tested anxiety-related behaviors. They found that knocking down Crhbp was anxiogenic in males and not in females. This was a critical experiment, but the shRNA constructs targeting Crhbp were validated solely in a cell line. It would have been more appropriate to perform a western blot on mPFC punches of adult mice, showing whether this lentiviral construct knocked down Crhbp expression in the mouse brain prior to behavioral testing. In fact, it also would have been useful to see a quantification of the shRNA transfection rate, as well as its specificity in vivo. As stated above, we also do not know the distribution of behavioral responses here either. Without these pieces of information, it is difficult to assess how reliable or robust their knockdown was.

The authors concluded that sexually dimorphic hormones act through the otherwise sexually monomorphic OxtrINs to regulate anxiety-related behaviors in males and sociosexual behaviors in females. We agree that OxtrINs interact with oxytocin and Crh to bring about sex-specific phenotypes, but we also think that using additional paradigms testing anxiety and social behaviors, such as a predator odor, novelty-suppressed feeding or social grooming, could shed more light on the nuances of mPFC circuitry. In addition, the authors suggested that OxtrINs are sexually monomorphic because they are equally abundant in males and females. The authors’ TRAP data however suggested that OxtrINs of males and females have different gene expression profiles (Table S2), thus indicating that these interneurons may form different connections in each sex that mediate the electrophysiological and behavioral differences we see in this study.

It would be interesting to overexpress Crhbp in female mice, preferably in a cell-type-specific manner, to see whether female mice would demonstrate the anxiety-like behavior seen in males. If the Crh:Crhbp balance is in fact mediating this sexually dimorphic behavior through OxtrINs, we would expect that doing these manipulations may “masculinize” the females’ behavior. Regardless, we believe that this study opens opportunities for future work into how oxytocin and Crh release from the hypothalamus may act together to coordinate behavior. It will also be interesting to see if single-cell RNA sequencing could provide insight into whether OxtrINs can be further divided into sexually dimorphic subtypes. As the authors pointed out, understanding the dynamics of Crh and oxytocin in the mPFC will be important for gender-specific therapy and treatment.

[1] Nakajima, M. et al. Oxytocin modulates female sociosexual behavior through a specific class of prefrontal cortical interneurons. Cell. 159, 295-305 (2014).

[2] Rungta, R. L. et al. Light controls cerebral blood flow in naïve animals. Nature Communications. 8, 14191 (2017).

[3] Heiman, M. et al. Cell-type-specific mRNA purification by translating ribosome affinity purification (TRAP). Nature Protocols. 9, 1282-1291 (2014).

On 2017 Aug 08, Christopher Tench commented:

Could you possibly provide the coordinates analysed otherwise it is difficult to interpret the results.

On 2016 Oct 21, Lydia Maniatis commented:

According to Kingdom: "A longstanding issue in vision research concerns whether the internal noise involved in contrast transduction is fixed or variable in relation to contrast magnitude."

This statement is precisely analogous to saying: A longstanding problem in chemistry is whether phlogiston is evenly or unevenly distributed in relation to object density.

The notion of "internal noise" is crude, lumping together every element of the visual process between light hitting the retina and the conscious percept. It is flatly inconsistent with perceptual experience, which is in no way "noisy," yet most proponents of this view would have us accept that the conscious percept directly reflects "low-level" and noisy spiking activity of individual or sets of neurons. In any event, no attempt has ever been made to corroborate the noise assumption.

It is not even clear what the criteria would be for corroboration on the basis of measurements at the physiological level. It would have to be shown, presumably, that identical "sensory inputs" produce a range and distribution of neural responses, this range and distribution being somewhat predictable; however "inputs" to brain activity don't come only from the external receptor organs, no matter how well we might be able to control these. Even if we could (inconceivably) control inputs perfectly, and even if we were able to say that (as is often claimed) at V1 neural responses are noisy, we would have to explain why this noise doesn't affect the conscious percept (which, again, is very stable) and yet is detectable on the basis of conscious experience. Graham (1992; 2011) has adopted the hypothesis that under certain conditions the brain becomes "transparent" so that the activities at lower levels of the processing hierarchy are act directly on the percept. It should be reasonably clear that such a view isn't worth entertaining, but if one wants to entertain it there are massive theoretical difficulties to overcome. It seems to imply that feedback and feedforward processes for some reason are frozen and some alternative, direct pathway to consciousness exists, all while other pathways are still active (because the inference generally applies to a discontinuity on a screen in a room, all of which are maintained in perception.)

Not surprisingly given the concept's vagueness, the case for "internal noise" has never been credibly made. But it is widely accepted.

Those who simply accept the internal noise assumption "measure internal noise" by analyzing simple "detection and discrimination" datasets on the basis of multiple layers of untested, untestable, or empirically untenable assumptions rolled into "computational models" including indispensable, multiple free parameters. (For a detailed examination of this technique, see PubPeer comments on Pelli (1985)).

In the absence of clear and explicit assumptions, relevant confounds remain unspecified, and tests, as here, are always ad hoc, hinging on particular datasets, and counting up "successes" as though by adding these together, they can outweigh unexplained failures. But failures are dispositive, of course, when we are aiming at a general explanation.

On 2017 Aug 20, Daniel Weiss commented:

The central problem with current standards of Lyme disease diagnosis and treatment is the absence of a highly sensitive and specific test, a gold standard, that can prove the presence of disease and/or demonstrate cure. When a patient complains of unremitting neurological and/or musculo-skeletal symptoms after completing a treatment, persistent infection is the most logical interpretation. Recent microbiological research has uncovered persistence mechanisms in virtually every prokaryotic organism(Conlon, Rowe, & Lewis, 2015 Advances in experimental medicine and biology; Harms, Maisonneuve, & Gerdes, 2016 Science,; Lewis & Shan, 2016 Molecular cell). It is not surprising that a bacterial species adapted to survive in multiple vertebrate hosts, and through multiple stages of the three-year cycle of its invertebrate host, might persist after antibiotic treatment(Feng, Shi, Zhang, & Zhang, 2015 Emerging microbes & infections). Borrelia species are characterized by immense plasticity in their expression of morphology, antigens, and genes.

In the laboratory, and in infected humans, antibiotics predictably induce the persister phenotype(Bijaya Sharma, Autumn V Brown, Nicole E Matluck, Linden T Hu, & Kim Lewis, 2015 Emerging microbes & infections).

Exposed to antibiotics, Borrelia burgdorferi rapidly loses the morphology of “active” motile, dividing spirochetes(Sapi et al., 2016 Int J Med Sci; Timmaraju et al., 2015 FEMS Microbiol Lett). The organism settles into dormancy in biofilm or as round bodies(Merilainen, Herranen, Schwarzbach, & Gilbert, 2015). Yet, it retains antigenicity and the genetic capacity to return to the spirochete form(Merilainen, Brander, Herranen, Schwarzbach, & Gilbert, 2016 Microbiology).

The failure to respond to a particular antibiotic regimen does not equate with "there is no infection present". The more appropriate conclusion may be that the antibiotic is ineffective for this infection. It is completely unclear whether these patients with presumed autoimmune disorders have persistent infection with B. burgdorferi.

On 2017 May 21, Misha Koksharov commented:

To clarify which of the thermostable luciferase mutants was used here (I've made a lot of them in L. mingrelica & P. pyralis Flucs - some are published, some are not):

LMLucR = the mutant G216N,A217L,S398M in Luciola mingrelica firefly luciferase (Koksharov MI, 2011).

On 2017 Feb 09, K Hollevoet commented:

Intramuscular antibody gene transfer as a means for prolonged in vivo antibody expression continues to gain traction as an alternative to conventional antibody production and delivery. The study by Kim et al. presents yet another elegant example of said approach, specifically expressing the anti-HER2 4D5 monoclonal antibody (mAb) in mice via plasmid-based gene electrotransfer. The authors report an average 4D5 peak concentration of up to 152 μg ml−1 in BALB/c mice, two weeks after intramuscular electrotransfer of the 4D5-encoding plasmid DNA (pDNA). mAb levels remained above 120 μg ml−1 for at least a month, as depicted in Figure 3d of the manuscript.¹ These results raise some questions, which, in our opinion, are not sufficiently addressed in the manuscript discussion.

Firstly, plasmid-based antibody gene electrotransfer in mice typically results in mere single-digit μg ml−1 mAb serum levels.² After careful consideration, we found no novelties in pDNA design, optimization or delivery in Kim et al.¹ that could explain their quantum leap in attained mAb titers – up to two log higher than the current available literature.

Secondly, the presented data by Kim et al.¹ surpass the expression levels of viral-based anti-HER2 antibody gene transfer studies in mice, with reported peak 4D5 and trastuzumab concentrations of 30 to 40 μg ml–1.^3,4 This further adds to the surprise, given viral vectors consistently outperform plasmid electrotransfer in terms of transgene expression.²

Thirdly, Figure 4f shows an average 4D5 serum concentration of 3.8 μg ml–1 in athymic nude mice, 22 days after tumor cell injection and, so we assume, approximately two weeks after pDNA delivery.¹ mAb titers in these tumor-bearing mice are thus about 40-fold lower than those in the BALB/c mice. Given identical dosing and delivery conditions were applied, the reason for this discrepancy is unclear. The difference in mAb titers appears too high to e.g. attribute it to inter-experiment or mice strain variability. Target-mediated drug disposition in the tumor-bearing mice, i.e. the binding of 4D5 to HER2, is also unlikely to have such impact, given the continuous and robust in vivo mAb production the authors found.

In conclusion, prolonged in vivo mAb expression above 100 μg ml–1 is unprecedented in non-viral antibody gene transfer. The impact of these findings, however, is mortgaged by the lack of explanation Kim et al. provide on the obvious differences with the available literature and with their own subsequent results – as outlined earlier. To allow for these remarkable findings to advance the field, we respectfully invite the authors to address the above concerns, and provide additional support for their data.

References: 1. Kim H, Danishmalik SN, Hwang H, Sin JI, Oh J, Cho Y, et al. Gene therapy using plasmid DNA-encoded anti-HER2 antibody for cancers that overexpress HER2. Cancer Gene Ther 2016 doi: 10.1038/cgt.2016.37. 2. Suscovich TJ, Alter G. In situ production of therapeutic monoclonal antibodies. Expert Rev Vaccines 2015; 14: 205-19. 3. Jiang M, Shi W, Zhang Q, Wang X, Guo M, Cui Z, et al. Gene therapy using adenovirus-mediated full-length anti-HER-2 antibody for HER-2 overexpression cancers. Clin Cancer Res 2006; 12: 6179-6185. 4. Wang G, Qiu J, Wang R, Krause A, Boyer JL, Hackett NR, et al. Persistent expression of biologically active anti-HER2 antibody by AAVrh.10-mediated gene transfer. Cancer Gene Ther 2010; 17: 559-570.

EDIT: A correction of the manuscript by the authors is ongoing based on the above comments. While awaiting the revision, our comments remain posted.

On 2016 Nov 27, Michael Gorn commented:

Thank you for creating this great video tutorial of our lumbar puncture (LP) technique. Video 1 is the most accurate representation of the way we performed the sonography for the study. You have captured the pertinent landmarks, especially the vascular supply of the anterior spinal space that we postulated was the main reason behind a bloody LP. This is how the concept of the Maximum Safe Depth (MSD) was developed. As you clearly demonstrated, the MSD measurements are very close in both transverse and longitudinal views, thus we used longitudinal views for convenience. Once the MSD measurement is obtained, it is marked on the needle as a safe entry depth while performing the LP. The MSD may be exceeded with caution if the needle entry level is shallow as justified by the Pythagorean equation demonstrated in the paper. However, if the entry angle is close to 90 degrees, we recommend redirecting the needle or reattempting the LP. Thank you again for your contribution.

On 2016 Nov 26, James Tsung commented:

Link to Video: http://bit.ly/2gK9osv

On 2016 Sep 16, Virginia Barbour commented:

As the Chair of COPE, I am writing to respond to the recent remarks of Dr. Horton with respect to the role and actions of COPE that he commented on in his Offline column1 highlighting the statin review by Professor Collins and colleagues, both of which were published in the issue of the 10th September. I also submitted this letter to the Lancet directly on 12th September.

COPE is an international interdisciplinary organisation, not just a UK one, whose remit is the provision of education and advice to members in questions related to publication ethics. We do have a process whereby an individual can bring to our attention complaints about journal processes, but we cannot interfere in editorial decisions and nor can we investigate the underlying issues of a complaint as we have neither the resources nor, more importantly, the appropriate level of subject–specific expertise. 

Dr Horton states that “COPE declined to act further”. This is incorrect. COPE did request details of processes at the BMJ, in accordance with our remit (http://publicationethics.org/contact-us). The guidance issued from COPE's review (I was not part of this final part of the process, having recused myself during the process because of the development of a potential conflict of interest) offered constructive criticism about how the BMJ had managed the peer review process. The BMJ had already addressed those issues following their own independent review and COPE was satisfied with the procedural changes that were implemented. 

As it is certainly not appropriate for COPE to make any specific judgment about effects on public health, COPE also recommended that Professor Collins and colleagues engaged in open dialogue on the specific issues in the medical literature. We note this has now happened with the publication of their review in The Lancet. 

Putting the correction of Dr Horton's record of events to one side, and instead looking for useful lessons, COPE would be interested in discussing Dr Horton's suggestion for an independent tribunal. It seems reasonable to assume that this tribunal would need public-funding and the ability to apply sanctions and, to a degree, become a regulator for the research community. This is not COPE's remit, but we would be interested in being part of the discussion on such an approach.

Virginia Barbour Chair, COPE

Competing interest. I'm the Chair of COPE. My decision to recuse myself from handling this issue midway through was because a colleague at PLOS (where I worked when the issue was brought to COPE) joined the BMJ.

Committee on Publication Ethics (COPE) cope_chair@publicationethics.org www.publicationethics.org

On 2016 Nov 11, James M Heilman commented:

A open access version can be read in full for free here http://journals.lww.com/academicmedicine/Abstract/publishahead/Why_Medical_Schools_Should_Embrace_Wikipedia__.98408.aspx

On 2016 Sep 26, Valter Silva commented:

This article discusses the Brazilian science based on a new resource for scientometrics called Nature Index, as well as the SJR. The 2012–2015 change in the main metric of Nature Index showed an increase of 18.9% for Brazil and currently is ranked 24th globally. From 1996 to 2015 (SJR) Brazilian science has produced more than 600 thousand citable papers, obtained more than 5 million citations, having over 400 papers with at least 400 citations and is responsible by half of Latin America publication output. Despite such numbers, there are flows in its internationalization. Much of the Brazilian science is produced by graduate students and professors gazetted, since the profession of scientist in Brazil is not a recognized position by the Ministry of Labor and Employment.

On 2016 Sep 15, Erick H Turner commented:

In addition to the two cited papers, close variations on this idea have been proposed in the following papers:

1 Walster GW, Cleary TA. A Proposal for a New Editorial Policy in the Social Sciences. The American Statistician 1970;24:16–9. doi:10.1080/00031305.1970.10478884

2 Newcombe RG. Towards a reduction in publication bias. Br Med J (Clin Res Ed) 1987;295:656–9.

3 Sridharan L, Greenland P. Editorial policies and publication bias: the importance of negative studies. Arch Intern Med 2009;169:1022–3. doi:10.1001/archinternmed.2009.100

4 Colom F, Vieta E. The need for publishing the silent evidence from negative trials. Acta Psychiatr Scand. 2011;123:91–4. doi:10.1111/j.1600-0447.2010.01650.x

5 Mirkin JN, Bach PB. Outcome-blinded peer review. Arch Intern Med 2011;171:1213–4–authorreply1214. doi:10.1001/archinternmed.2011.56

6 Turner EH. Publication bias, with a focus on psychiatry: causes and solutions. CNS Drugs 2013;27:457–68. doi:10.1007/s40263-013-0067-9

7 Smulders YM. A two-step manuscript submission process can reduce publication bias. J Clin Epidemiol Published Online First: July 2013. doi:10.1016/j.jclinepi.2013.03.023

On 2016 Nov 26, David Keller commented:

Thank you. Your findings compare with the CALM-PD study [1], which found "For subjects who consumed >12 ounces of coffee/day, the adjusted hazard ratio for the development of dyskinesia was 0.61 (95% CI, 0.37-1.01) compared with subjects who consumed <4 ounces/day." in patients at an early stage of PD. Longer follow-up should indeed be helpful in assessing whether the benefits of increased caffeine ingestion are durable, at what cost in side-effects, and whether higher doses of caffeine provide correspondingly higher benefits.

Reference

1: Wills AM, Eberly S, Tennis M, Lang AE, Messing S, Togasaki D, Tanner CM, Kamp C, Chen JF, Oakes D, McDermott MP, Schwarzschild MA; Parkinson Study Group. Caffeine consumption and risk of dyskinesia in CALM-PD. Mov Disord. 2013 Mar;28(3):380-3. doi: 10.1002/mds.25319. PubMed PMID: 23339054; PubMed Central PMCID: PMC3608707.

On 2016 Nov 25, Marcello Moccia commented:

We agree with the need for evaluating incidence and severity of dyskinesia in the long-term assessment of drug efficacy in PD. However, studies including de novo PD patients have to consider early markers of motor progression which indeed are associated with the development of dyskinesia in the long term. In view of this, we showed that the voluptuary consumption of caffeine-containing products is associated with reduced need for levo-dopa and with reduced accrual of motor symptoms. Of course, a longer follow-up will possibly confirm the positive impact of caffeine use also on dyskinesia.

On 2016 Nov 09, David Keller commented:

What effect did caffeine consumption have on the incidence or severity of dyskinesia?

In this study, higher caffeine consumption was associated with a lower rate of starting levodopa treatment and reduced motor and non-motor disability. Any treatment for Parkinson disease (PD) which delays or decreases the need for levodopa therapy should be evaluated for its propensity to hasten the onset of dyskinesia, or to worsen established dyskinesia. If the motor and non-motor benefits of caffeine are accompanied by a risk of developing dyskinesia equal to that of a levodopa regimen with equivalent benefits, then it is unclear why ingesting caffeine on a pharmacologic basis is preferable to simply initiating levodopa when it is needed.

On 2016 Sep 15, Gary Goldman commented:

Civen et al report the average HZ incidence of 12.8 cases/100,000 children aged <10 years during 2007 to 2010.¹ Since two different ascertainment sources are available for the reporting of HZ cases—schools (including preschools), and public and private healthcare providers (including hospitals)—capture-recapture techniques could have been employed to determine that the Antelope Valley project had approximately 50% case ascertainment, and thus, the true HZ figure is approximately double that reported. Interestingly, 25.6 cases/100,000, or twice the reported rate closely compares with the rate of 27.4 cases/100,000 (95% C.I. 22.7-32.7) based on 172,163 vaccinated children with overall follow-up of 446,027 person-years among children aged <12 years during 2007-2008 reported by Tseng et al.²

Civen et al report that among 10- to 19-year olds a 63% increasing trend in HZ incidence from 2000 to 2006 was documented; however, “the increased incidence could not be confidently explained.” The authors concede, “the possibility persists that children infected by wild-type VZV experienced increased rates of HZ because they were having fewer opportunities to be exposed to exogenous VZV, leading to reduced immune control of HZ.“¹ The reason that the 63% increasing trend in HZ incidence among 10- to 19-year olds has not been confidently explained is that the methodology utilized by Civen et al did not include stratifying HZ incidence rates by using two widely different cohorts: those vaccinated and those with a history of wild-type (natural) varicella. Computing a single mean HZ incidence rate of a bimodal distribution is statistically invalid and conceals the reality that the HZ incidence rate among children and adolescents with a history of wild-type varicella has an increasing trend.³ By performing such a stratified analysis, Civen et al could have tested the hypothesis that individuals with a prior history of varicella are experiencing increasing HZ incidence due to fewer exogenous exposures, and thus, reduced opportunities for boosting cell-mediated immunity to VZV.^4,5

Civen et al state, “The case for this hypothesis has weakened as studies have found no acceleration in rates of HZ among adults in the United States since the varicella vaccination was introduced, despite the fact that opportunities for varicella exposure have plummeted.” Interestingly, the same Antelope Valley surveillance project did collect HZ cases during 2000-2001 and 2006-2007 that showed statistically significant increases among adults. The Antelope Valley annual summary to the CDC demonstrates that in 2000 and 2001, HZ cases (not ascertainment corrected) reported to the project either maintained or increased in every adult 10-year age category (20–29, 30–39, . . . , 60–69 years), yielding a statistically significant difference. Reported HZ cases among adults aged 20–69 years increased 28.5%—from 158 in 2000 to 203 in 2001 (p <0.042; t = 2.95, df = 4).⁶ Again, HZ incidence rates among adults aged 50 years and older increased from 390/100,000 p-y in 2006 to 470/100,000 p-y in 2007 with a statistically significant rate ratio of 1.2 (95% CI: 1.04–1.40).⁷ A Canadian study by Marra et al concludes that "the incidence of zoster and PHN is increasing with time" and suggests "recent studies have shown an increasing incidence of herpes zoster infection, which may be related to the introduction of varicella vaccination programs in children."⁸

The United States has traded a dramatic reduction in varicella disease which in the prevaccine era accounted for only 25% of the VZV medical costs (i.e., 75% of VZV medical costs were attributed to cases of HZ) for a disproportional increase in HZ costs associated with increasing HZ incidence among adults with a history of wild-type varicella. It is an unfortunate fact that 20 years after the introduction of the varicella vaccine in the US, healthcare officials are still claiming that the mechanism of exogenous boosting “is not well understood” and “the case for this hypothesis has weakened,” when in reality, the data currently exist to understand this biological mechanism first proposed in 1965 by Dr. Robert Edgar Hope-Simpson.⁴ "Rather than eliminating varicella in children as promised, routine vaccination against varicella has proven extremely costly and has created continual cycles of treatment and disease."³

References

[1] Civen R, Marin M. Zhang J. Abraham A, Harpaz R, Mascola L. Bialek S. Update on incidence of herpes zoster among children and adolescents after implementation of varicella vaccination, Antelope Valley, CA, 2000 to 2010. Pediatr Infect Dis J. 2016 Oct; 35(10):1132-1136.Civen R, 2016

[2] Tseng HF, Smith N, Marcy SM, Sy LS, Jacobsen SJ. Incidence of herpes zoster among children vaccinated with varicella vaccine in a prepaid health care plan in the United States, 2007, 2008. Pediatr Infect Dis J 2009;28(December(12)):1069–72.Tseng HF, 2009

[3] Goldman GS and King PG. Review of the United States universal varicella vaccination program: Herpes zoster incidence rates, cost-effectiveness, and vaccine efficacy based primarily on the Antelope Valley Varicella active surveillance project data. Vaccine 2013; 31(13): 1680–1694.Goldman GS, 2013

[4] Hope-Simpson RE. The nature of herpes zoster: a long term study and a new hypothesis. Proc R Soc Med 1965; 58: 9–20.HOPE-SIMPSON RE, 1965

[5] Guzzetta G, Poletti P, Del Fava E, Ajelli M, Scalia Tomba GP, Merler S, et al. Hope-Simpson’s progressive immunity hypothesis as a possible explanation for Herpes zoster incidence data. Am J Epidemiol 2013; 177(10): 1134–1142.Guzzetta G, 2013

[6] Maupin T, Peterson C, Civen R and Mascola L. Varicella Active Surveillance Project (VASP). 2000, 2001, Annual Summary. Antelope Valley, County of Los Angeles Department of Health Services (LADHS), Acute Communicable Disease Control, Centers for Disease Control and Prevention (CDC) Cooperative Agreement No. U66/CCU911165-10.

[7] Maupin T, Peterson C, Civen R and Mascola L. Varicella Active Surveillance Project (VASP). 2006, 2007 Annual Summary. Antelope Valley , County of Los Angeles Department of Health Services (LADHS), Acute Communicable Disease Control, Centers for Disease Control and Prevention (CDC) Cooperative Agreement No. 5U01 IP000020-02/5U01 IP000020-04.

[8] Marra F, Chong M and Najafzadeh M. Increasing incidence associated with herpes zoster infection in British Columbia, Canada. BMC Infect Dis. 2016 Oct 20; 16(1):589.Marra F, 2016

On 2017 May 31, Sergio Uribe commented:

Timely and necessary reflection.

On 2016 Dec 10, Arnaud Chiolero MD PhD commented:

These findings were, unfortunately, expected. They suggest that systematic reviews should not be always regarded as the highest level of evidence; it is evident that they can be seriously biased - like any other studies.

On 2016 Nov 22, Natalie Parletta commented:

Good point re null results less likely to be published Matthew Romo. I think we also need to consider that in some instances there is a vested interest in publishing null findings, such as the systematic review on omega-3 fatty acids and cardiovascular disease (BMJ 2006; 332 doi: http://dx.doi.org/10.1136/bmj.38755.366331.2F) which did not include positive studies before 2000 (which had led to recommendations to eat fish/take fish oil for CVD) and has been critiqued for serious methodological flaws (https://www.cambridge.org/core/journals/british-journal-of-nutrition/article/pitfalls-in-the-use-of-randomised-controlled-trials-for-fish-oil-studies-with-cardiac-patients/65DDE2BD0B260D1CF942D1FF9D903239; http://www.issfal.org/statements/hooper-rebuttable). Incidentally I learned that the journal that published one of the null studies sold 900,000 reprints to a pharmaceutical company (that presumably sells statins).

On 2016 Nov 04, Matthew Romo commented:

Thank you for this very thought provoking paper. With the skyrocketing amount of systematic reviews (and meta-analyses) published, I wonder how many did not identify any evidence for their research question. If systematic reviews are research, shouldn’t we expect null results, at least once in a while? Quantifying the relative number of systematic reviews with null results (which seem to be very few) might be helpful in further understanding the degree of bias there is in published systematic reviews. After all, research should be published based on the importance of the question they seek to answer and their methodological soundness, rather than their results (Greenwald, 1993).

"Null" systematic reviews that find no evidence can be very informative for researchers, clinicians, and patients, provided that the systematic review authors leave no stone unturned in their search, as they ought to for any systematic review. For researchers, they scientifically identify important gaps in knowledge where future research is needed. For clinicians and patients, they can provide an understanding of practices that don’t have a reliable evidence base. As stated quite appropriately by Alderson and Roberts in 2000, “we should be willing to admit that ‘we don’t know’ so the evidential base of health care can be improved for future generation.”

Matthew Romo, PharmD, MPH Graduate School of Public Health and Health Policy, City University of New York

Alderson P, Roberts I. Should journals publish systematic reviews that find no evidence to guide practice? Examples from injury research. BMJ. 2000;320:376-377.

Greenwald AG. Consequences of prejudice against the null hypothesis. In: A Handbook for Data Analysis in the Behavioural Sciences, edited by Keren G, Lewis C, Hillsdale, NJ, Lawrence Erlbaum, 1993, pp419–448.

On 2016 Dec 30, Arturo Martí-Carvajal commented:

What is the degree of responsibility of either editor-in-chief of journal or peer reviewers in the publication of systematic reviews with doubtful quality?

On 2016 Sep 18, Hilda Bastian commented:

Thanks, John - that's as close we'll get, and we do agree on far more than we disagree, as ever.

I agree we should face the data, and be meticulous about it. I just don't agree that indexing has the same effect on a tagged category as it has for a filter: especially not when the filter is so broad that it encompasses the variety of terms people use to describe their work. I remain convinced that the appropriate time trend comparators are filter to filter, with triangulation of sources. I don't think it's highly likely that 90% of the RCTs are in the first 35% of tagged literature.

I don't think people should hold off publishing a systematic review that was done before deciding to fund or run a trial, until a report of the trial or its methods is published - and ideally, they would be done by different people. Intellectual conflicts of interest can be as powerful as any other. And I don't think that trialists interpreting what their trial means in the context of other evidence meets the criterion, unconflicted. Nor do I think the only systematic reviews we need are those with RCTs.

I don't think Cochrane reviews are all good quality and unconflicted - in fact, the example of a conflicted review with quality issues in my comment was a Cochrane review. I agree there is no prestigious name that guarantees quality. (It's a long time since I left the Cochrane Collaboration, by the way.) My comments aren't because I disagree that there is a flood of bad quality "systematic" reviews and meta-analyses: the title of your article is one of the many things I agree with. See for example here, here, and quite a few of my comments on PubMed Commons.

But the main reason for this reply is to add into this stream the reason I feel some grounds for optimism about something else we would both fervently agree on: the need to chip away at the problem of extensive under-reporting of clinical trials. As of January 2017, the mechanisms and incentives for reporting a large chunk of trials - those funded by NIH and affected by the FDA's scope - will change (NIH, 2016). Regardless of what happens with synthesis studies, any substantial uptick in trial reporting would be great news.

On 2016 Sep 18, John Ioannidis commented:

Dear Hilda,

thank you for all these wise thoughts. Based on prior experience, at this point in time (mid-September) the numbers for "Type of Article" meta-analysis, systematic reviews and randomized controlled trial for 2015 are likely to increase by about 10% with more complete indexing. I have taken this into account in my calculations.

I fully agree with Iain Chalmers that every trial should start and finish with a systematic review. I fervently defend upfront this concept in my paper when I say that "it is irrational not to systematically review what is already known before deciding to perform any new study. Moreover, once a new study is completed, it is useful to update the cumulative evidence", even specifically citing Iain's work. But the publication of these systematic reviews are (and should be) integral with the publication of the specific new studies. I have not counted separately the systematic reviews that are embedded within trial publications. If I were to do this, then the numbers of systematic reviews would be even higher. My proposal even goes a step further in arguing that systematic reviews and meta-analyses should be even more tightly integrated with the primary studies. Meta-analyses should become THE primary studies par excellence.

So, the answer to your question "But in an ideal world, isn't a greater number of systematic reviews than RCTs just the way it should be?" the answer is clearly "No", if we are taking about the dominant paradigm of systematic reviews of low quality that are done in isolation of the primary evidence and represent a parallel universe serving mostly its own conflicts. The vast majority of the currently published systematic reviews are not high-quality, meticulous efforts, e.g. Cochrane reviews, and they are entirely disjoint from primary studies. Cochrane reviews represent unfortunately less than 5% of this massive production. While I see that you and some other Cochrane friends have felt uneasy with the title of my paper and this has resulted in some friendly fire, I ask you to please look more carefully at this threatening pandemic which is evolving in the systematic review and meta-analysis world. Even though I trust that Cochrane is characterized by well-intentioned, non-conflicted and meticulous efforts, this bubble, which is 20-50 times larger than Cochrane, is growing next door. Let us please face the data, recognize this major problem and not try to defend ANY systematic reviews and meta-analyses as if they have value no matter what just because they happen to carry such a prestigious name.

On 2016 Sep 18, Hilda Bastian commented:

Thanks, John, for taking this so seriously - that's extremely helpful and I certainly agree with you that the rate of publication of systematic reviews is growing faster than RCTs. So the point you are talking about may well be reached at some point. Unless the rate of growth equalizes, and unless the rate of RCTs that are unpublished drops substantially: and both of those remain possible.

This comparison is much better, but it can't solve the underlying issues. Human indexing resources did not increase exponentially along with the exponential increase of literature. As of today, searching for PubMed records with 2015 in the PubMed entry date [EDAT], only 35% also have a 2015 date for completed indexing [DCOM] (which from PubMed Help looks to me the way you would check for that - but an information specialist may correct me here). That's roughly what I would expect to see: individually indexing well over a million records a year is a colossal undertaking. Being finished 2015 in just a few months while 2016 priorities are pouring in would be amazing. And we know that no process of prioritizing journals will solve this problem for trials, because the scatter across journals is so great (Hoffmann T, 2012).

So any comparison between a tagged set (RCTs) and a search based on a filter with text words (which includes systematic review or meta-analysis in the title or abstract), could generate potentially very biased estimates, no matter how carefully the results are analyzed. And good systematic reviews of non-randomized clinical trials, and indeed, other methodologies - such as systematic reviews of adverse events, qualitative studies, and more - are valuable too. Many systematic reviews would be "empty" of RCTs, but that doesn't make them useless by definition.

I couldn't agree with you more enthusiastically, though, that we still need more, not fewer, well-done RCTs, systematic reviews, and meta-analyses by non-conflicted scientists. I do add a caveat though, when it comes to RCTs. RCTs are human experimentation. It is not just that they are resource-intensive: unnecessary RCTs and some of the ways that RCTs can be "bad", can cause direct harm to participants, in a way that an unnecessary systematic review cannot. The constraints on RCTs are greater: so they need to be done on questions that matter the most and where they can genuinely provide better information. If good enough information can come from systematically reviewing other types of research, then that's a better use of scarce resources. And if only so many RCTs can be done, then we need to be sure we do the "right" ones.

For over 20 years, Iain Chalmers has argued that an RCT should not be done without a systematic review to show the RCT is justified - and there should be an update afterwards. Six years ago - he, Mike Clarke and Sally Hopewell concluded that we were nowhere near achieving that (Clarke M, 2010). The point you make about the waste in systematic reviewing underscores that point, too. But in the ideal world, isn't a greater number of systematic reviews than RCTs just the way it should be?

On 2016 Sep 17, John Ioannidis commented:

Dear Hilda,

Thank you for your follow-up comment on my reply, I always cherish your insights. I tried to get a more direct answer to the question on which we both have some residual uncertainty, i.e. whether currently published systematic reviews of trials outnumber new randomized controlled trials. So, I collected more data.

First, while we can disagree on some minor technical details, it is very clear that the annual rate has been increasing extremely fast for “meta-analyses” and very fast for “systematic reviews”, while it is rising slowly for “randomized controlled trials” types of articles. In a search as of today, the numbers per year between 2009 and 2015 using the “type of article” searches (with all their limitations) are 3243-3934-4858-6570-8192-9632-9745 for meta-analysis, 15085-17353-19378-22575-25642-29261-31609 for systematic reviews and 17879-18907-20451-22339-24538-24459-22066 for randomized controlled trials. The data are not fully complete for 2015 given that “type of article” assignments may have some delay, but comparing 2014 versus 2009 where the data are unlikely to change meaningfully with more tags, within 5-years the rate of publication of meta-analyses tripled, the rate of publication of systematic reviews doubled, while the rate of publication of randomized trials increased by only 36% (almost perfectly tracking the 33% growth of total PubMed items in the same period).

Type of article is of course not perfectly sensitive or specific in searching. So, I took a more in-depth look in a sample of 111 articles that are Type of article=“randomized controlled trial” among the 22066 published in 2015 (in the order of being retrieved by a 2015 [DP] search selecting the first and every 200th afterwards, i.e. 1, 201, 401, etc). Of the 111, 17 represent secondary analyses (the majority of secondary analyses of RCTs are not tagged as “randomized controlled trial”), 5 are protocols without results, 6 are non-human randomized studies (on cattle, barramundi etc), and 12 are not randomized trials, leaving a maximum of 71 new randomized controlled trials. I say “maximum”, because some of those 71 may actually not be randomized (e.g. there is a substantial number of “randomized” trials from China and past in-depth evaluations have shown that many/most are not really randomized even they say they are) and some others may also be secondary or duplicate publications but this is not easy to decipher based on this isolated sampling. Even if 71/111 are new RCTs, this translates to (71/111)x22060=14114 new RCTs (or articles masquerading as new RCTs) in 2015. Allowing for some missed RCTs and not yet tagged ones, it is possible that the number of new RCTs published currently is in the range of 15,000 per year. Of the 71 studies that were new RCTs or masquerading as such, only 25 had over 100 randomized participants and only 1 had over 1000 randomized participants. Clinically informative RCTs are sadly very few.

I also examined the studies tagged as Type of Article “meta-analysis” or “systematic review” or “review” published in 2015 [DP], combined with (trial* OR treatment* OR randomi*). Of the 49,166 items, I selected 84 for in-depth scrutiny (the first and every 600th afterwards, i.e. 1, 601, 1201, etc). Overall, 30 of the 84 were systematic reviews and/or meta-analyses of trials or might be masquerading as such to the average reader, i.e. had some allusion to search databases and/or search strategies and/or systematic tabulation of information. None of these 30 are affected by any of the potential caveats you raised (protocols, ACP Journal Club, split reviews, etc). Extrapolating to the total 49166, one estimates 17988 systematic reviews and/or meta-analyses of trials (or masquerading as such) in 2015. Again, allowing for missed items (e.g. pooled analyses of multiple trials conducted by the industry are not tagged as such Types of Articles), for those not yet tagged, and for a more rapid growth for such studies in 2016 than for RCTs, it is likely that the number of systematic reviews and/or meta-analyses of trials published currently is approaching 20,000 per year. If the criteria for “systematic review of trials” become more stringent (as in Page et al, 2016), this number will be substantially smaller, but will still be quite competitive against the number of new RCTs. Of course, if we focus on both stringent criteria and high quality, the numbers drop precipitously, as it happens also with RCTs.

I am sure that these analyses can be done in more detail. However, the main message is unlikely to change. There is a factory of RCTs and a far more rapidly expanding factory of systematic reviews and meta-analyses. The majority of the products of both factories are useless, conflicted, misleading or all of the above. The same applies to systematic reviews and meta-analyses for most other types of study designs in biomedical research. This does not mean that RCTs, systematic reviews, and meta-analyses are not a superb idea. If well done by non-conflicted scientists, they can provide the best evidence. We need more, not fewer, such studies that are well done and non-conflicted.

On 2016 Sep 16, Hilda Bastian commented:

Thanks, John, for the reply - and for giving us all so much to think about, as usual!

I agree that there are meta-analyses without systematic reviews, but the tagged meta-analyses are included in the filter you used: they are not additional (NLM, 2016). It also includes meta-analysis in the title, guidelines, validation studies, and multiple other terms that add non-systematic reviews, and even non-reviews, to the results.

In Ebrahim S, 2016, 191 primary trials in only high impact journals were studied. Whether they are typical of all trials is not clear: it seems unlikely that they are. Either way, hundreds of reports for a single trial is far from common: half the trials in that sample had no secondary publications, only 8 had more more than 10, and none had more than 54. Multiple publications from a single trial can sometimes be on quite different questions, which might also need to be addressed in different systematic reviews.

The number of trials has not been increasing as fast as the number of systematic reviews, but the number has not reached a definite ongoing plateau either. I have posted an October 2015 update to the data using multiple ways to assess these trends in the paper by me, Paul Glasziou, and Iain Chalmers from 2010 (Bastian H, 2010) here. Trials have tended to fluctuate a little from year to year, but the overall trend is growth. As the obligation to report trials grows more stringent, the trend in publication may be materially affected.

Meanwhile, "systematic reviews" in the filter you used have not risen all that dramatically since February 2014. For the whole of 2014, there were 34,126 and in 2015 there were 36,017 (with 19,538 in the first half of 2016). It is not clear without detailed analysis what part of the collection of types of paper are responsible for that increase. The method used to support the conclusion here about systematic reviews of trials overtaking trials themselves was to restrict the systematic review filter to those mentioning trials or treatment - “trial* OR randomi* OR treatment*”. That does not mean the review is of randomized trials only: no randomized trial need be involved at all, and it doesn't have to be a review.

Certainly, if you set the number of sizable randomized trials high, there will be fewer of them than of all possible types of systematic review: but then, there might not be all that many very sizable, genuinely systematic reviews either - and not all systematic reviews are influential (or even noticed). And yes, there are reviews that are called systematic that aren't: but there are RCTs called randomized that aren't as well. What's more, an important response to the arrival of a sizeable RCT may well be an updated systematic review.

Double reports of systematic reviews are fairly common in the filter you used too, although far from half - and not more than 10. Still, the filter will be picking up protocols as well as their subsequent reviews, systematic reviews in both the article version and coverage in ACP Journal Club, the full text of systematic reviews via PubMed Health and their journal versions (and the ACP Journal Club coverage too), individual patient data analyses based on other systematic reviews, and splitting a single systematic review into multiple publications. The biggest issue remains, though, that as it is such a broad filter, casting its net so very wide across the evidence field, it's not an appropriate comparator for tagged sets, especially not in recent years.

On 2016 Sep 16, John Ioannidis commented:

Dear Hilda,

thank you for the very nice and insightful commentary on my article. I think that my statement "Currently, probably more systematic reviews of trials than new randomized trials are published annually" is probably correct. The quote of 8,000 systematic reviews in the Page et al. 2016 article is using very conservative criteria for systematic reviews and there are many more systematic reviews and meta-analyses, e.g. there is a factory of meta-analyses (even meta-analyses of individual level data) done by the industry combining data of several trials but with no explicit mention of systematic literature search. While many papers may fail to satisfy stringent criteria of being systematic in their searches or other methods, they still carry the title of "systematic reviews" and most readers other than a few methodologists trust them as such. Moreover, the 8,000 quote was from February 2014, i.e. over 2.5 years ago, and systematic reviews' and meta-analyses' publication rates rise geometrically. Conversely, there is no such major increase in the annual rate of published randomized controlled trials. Furthermore, the quote of 38,000 trials in the Cochrane database is misleading, because it includes both randomized and non-randomized trials and the latter may be the majority. Moreover, each randomized controlled trial may have anywhere up to hundreds of secondary publications. On average within less than 5 years of a randomized trial publication, there are 2.5 other secondary publications from the same trial (Ebrahim et al. 2016). Thus the number of published new randomized trials per year is likely to be smaller than the number of published systematic reviews and meta-analyses of randomized trials. Actually, if we also consider the fact that the large majority of randomized trials are small/very small and have little or no impact, while most systematic reviews are routinely surrounded by the awe of the "highest level of evidence", one might even say that the number of systematic reviews of trials published in 2016 is likely to be several times larger than the number of sizable randomized trials published in the same time frame.

On 2016 Sep 16, Hilda Bastian commented:

There are many important issues raised in this paper on which I strongly agree with John Ioannidis. There is a lot of research waste in meta-analyses and systematic reviews, and a flood of very low quality, and he points out the contributing factors clearly. However, there are some issues to be aware of in considering the analyses in this paper on the growth of these papers, and their growth in comparison with randomized and other clinical trials.

Although the author refers to PubMed's "tag" for systematic reviews, there is no tagging process for systematic reviews, as there is for meta-analyses and trials. Although "systematic review" is available as a choice under "article types", that option is a filtered search using Clinical Queries (PubMed Help), not a tagging of publication type. Comparing filtered results to tagged results is not comparing like with like in 2 critical ways.

Firstly, the proportion of non-systematic reviews in the filter is far higher than the proportion of non-meta-analyses and non-trials in the tagged results. And secondly, full tagging of publication types for MEDLINE/PubMed takes considerable time. When considering a recent year, the gulf between filtered and tagged results widens. For example, as of December 2015 when Ioannidis' searches were done, the tag identified 9,135 meta-analyses. Today (15 September 2016), the same search identifies 11,263. For the type randomized controlled trial, the number tagged increased from 23,133 in December to 29,118 today.

In the absence of tagging for systematic reviews, the more appropriate comparisons are using filters for both systematic reviews and trials as the base for trends, especially for a year as recent as 2014. Using the Clinical Queries filter for both systematic reviews and therapy trials (broad), for example, shows 34,126 for systematic reviews and 250,195 trials. Page and colleagues estimate there were perhaps 8,000 actual systematic reviews according to a fairly stringent definition (Page MJ, 2016) and the Centre for Reviews and Dissemination added just short of 9,000 systematic reviews to its database in 2014 (PubMed Health). So far, the Cochrane Collaboration has around 38,000 trials in its trials register for 2014 (searching on the word trial in CENTRAL externally).

The number of systematic reviews/meta-analyses has increased greatly, but not as dramatically as this paper's comparisons suggest, and the data do not tend to support the conclusion in the abstract here that "Currently, probably more systematic reviews of trials than new randomized trials are published annually".

Ioannidis suggests some bases for some reasonable duplication of systematic reviews - these are descriptive studies, with many subjective choices along the way. However, there is another critical reason that is not raised: the need for updates. This can be by the same group publishing a new version of a systematic review or by others. In areas with substantial questions and considerable ongoing research, multiple reviews are needed.

I strongly agree with the concerns raised about conflicted systematic reviews. In addition to the issues of manufacturer conflicts, it is important not to underestimate the extent of other kinds of bias (see for example my comment here). Realistically, though, conflicted reviews will continue, building in a need for additional reviewers to tackle the same ground.

Systematic reviews have found important homes in clinical practice guidelines, health technology assessment, and reimbursement decision-making for both public and private health insurance. But underuse of high quality systematic reviews remains a more significant problem than is addressed here. Even when a systematic review does not identify a strong basis in favor of one option or another, that can still be valuable for decision making - especially in the face of conflicted claims of superiority (and wishful thinking). However, systematic reviews are still not being used enough - especially in shaping subsequent research (see for example Habre C, 2014).

I agree with Ioannidis that collaborations working prospectively to keep a body of evidence up-to-date is an important direction to go - and it is encouraging that the living cumulative network meta-analysis has arrived (Créquit P, 2016). That direction was also highlighted in Page and Moher's accompanying editorial (Page MJ, 2016). However, I'm not so sure how much of a solution this is going to be. The experience of the Cochrane Collaboration suggests this is even harder than it seems. And consider how excited people were back in 1995 at the groundbreaking publication of the protocol for prospective, collaborative meta-analysis of statin trials (Anonymous, 1995) - and the continuing controversy that swirls, tornado-like, around it today (Godlee, 2016).

We need higher standards, and skills in critiquing the claims of systematic reviews and meta-analyses need to spread. Meta-analysis factories are a serious problem. But I still think the most critical issues we face are making systematic reviews quicker and more efficient to do, and to use good ones more effectively and thoroughly than we do now (Chalmers I, 2009, Tsafnat G, 2014).

Disclosure: I work on projects related to systematic reviews at the NCBI (National Center for Biotechnology Information, U.S. National Library of Medicine), including some aspects that relate to the inclusion of systematic reviews in PubMed. I co-authored a paper related to issues raised here several years ago (Bastian H, 2010), and was one of the founding members of the Cochrane Collaboration.

On 2016 Oct 24, Sean Ekins commented:

this poster provides some more details http://www.slideshare.net/ekinssean/starting-small-companies-focused-on-rare-diseases

On 2016 Oct 03, Andrey Alexeyenko commented:

The counts for 2x2 table seem to be wrong. The last two terms should be:

n12 | k - x

n22 | N - [ k + d ] + x

so that n11 + n12 + n21 + n22 sum up to all genes from the genome. That is why the table is called "contingency table".

On 2016 Dec 18, Laura Williams commented:

Our online journal club discussed this paper on October 27, 2016. https://www.youtube.com/watch?v=3IKyeaBk4qM

On 2016 Nov 22, Peter Hajek commented:

Thank you Laurie for looking into this and confirming that the data do not suggest that vaping undermines quitting. Regarding a possible benefit of vaping, a better test would be including participants who were smoking at 3M, as you did, but compare those who did and those who did not try vaping BETWEEN the 3M and 6M follow-up. This is because those still smoking and reporting using EC prior to the 3M f-u are self-selected for not benefiting from vaping (up to that point anyway). Doing it the way suggested above avoids some of that problem - but the result would remain affected by self-selection and uncertainty about the purpose and intensity of e-cig use. Thanks again, Peter

On 2016 Nov 21, Laurie Zawertailo commented:

We thank Dr. Hajek for his constructive criticism of our paper and his suggested alternate analysis. We agree that smokers reporting e-cigarette use at follow-up may have been less likely to be able to quit using the standard treatment offered and so may have resorted to e-cigarettes to aid in their quit attempt. We were able to conduct the suggested analysis by looking at smokers who were not quit at the 3-month follow-up time point (i.e. failed on the initial treatment, n=1626). At the 6-month follow-up we assessed whether or not they reported being quit and whether or not they had used e-cigarettes. At 6-month follow-up, 11.4% of e-cigarette non-users reported quit (7-day PPA), compared to 9.2% of e-cigarette users (p=0.24, NS). Therefore, there is no evidence to support Dr. Hajek’s hypothesis that e-cigarette use will increase quit rates among those who do not quit smoking using standard evidence-based treatment (NRT plus counselling). Again these data are limited due to the lack of information regarding dose and duration of e-cigarette use and due to bias caused by self-selection.

On 2016 Sep 27, Peter Hajek commented:

The conclusion is mistaken. People who failed in their initial attempt to quit smoking with NRT would be much more likely to try alternatives than those who quit successfully. The finding that non-EC use group did better is an artifact of this - treatment successes were concentrated there. It would be more informative to look at people who failed with the initial treatment and compare those who did and those who did not try e-cigarettes during the follow-up period. Such a comparison may well show that e-cigarette use had a positive effect. Self-selection would remain a problem, but perhaps the authors could check this?

On 2017 Oct 10, Shaun Khoo commented:

Green Open Access: The accepted manuscript of this review paper is available from the UNSW Institutional Repository.

On 2016 Sep 10, Seyed Moayed Alavian commented:

I read with interest this publication , the study has done in resistant cases and 40.2% were null-responders and 56.9% of them had liver cirrhosis. The result of SVR 99.0% is very interesting for scientists. It is very critical for us to understand the tolerability of patients to these regimens especially in liver cirrhosis patients?. And did they included the cirrhotic patients in child B and C in their study or not?

On 2016 Sep 19, Gustav van Niekerk commented:

We (van Niekerk G, 2016) have recently argued that sickness associated anorexia (SAA) may represent a strategy to maintain high levels of autophagic flux on a systemic level systemically. (Also see van Niekerk G, 2016 for an evolutionary perspective).

An upregulation of autophagy during an infection may be critical for a number of reasons:

• Serum and AA starvation induce autophagy in macrophages and protect against TB infection (Gutierrez MG, 2004).

• We speculate that hepatic autophagy may play a critical role in clearing LPS and bacteria rom circulation.

• Pathogens entering a cell must quickly subvert host processes to prevent being degraded by autophagy. In this regard, up regulation of autophagic flux would confront pathogens with a narrower window of opportunity to modulate host machinery. Thus, autophagy enhance cell autonomous defence.

• Autophagy processes ribosomal components into antimicrobial peptides (Ponpuak M, 2010). Note that all nucleated cells have ribosomes and are capable of autophagy, this suggesting that autophagy may again enhance cell-autonomous defence.

• Autophagy is also involved in the non-canonical expression of epitopes on MHC II by non-professional antigen presenting cells such as adipocytes, muscle and endothelium cells.

Autophagy may also be important in cell survival. As an example, tissue ischemia, the release of biocidal agents from immune cells as well as the increase in misfolded proteins resulting from a febrile response may lead to the generation of toxic protein aggregates. Here, autophagy may promote cell survival by processing ‘overflow’ of damage proteins aggregates when proteasome pathway is overwhelmed.

Fasting-induced autophagy may thus promote host tolerance and resistance.

On 2017 May 26, Kenneth Witwer commented:

This article has now been retracted:

https://www.nature.com/articles/srep46826

The authors also repeated some of their experiments with appropriate methods and reported, "we were unable to confirm specific amplification of these miRNAs in human blood. Thus, we were not able to validate the central hypothesis of this paper."

On 2016 Nov 02, Kenneth Witwer commented:

Following the previous comments, tweets on this subject raise a few more perceived issues:

https://twitter.com/ProfParrott/status/792472109834498049

https://twitter.com/ProfParrott/status/792472735427461120

Importantly, the manufacturer of the qPCR kit used in this study states that it is not for use with plant miRNAs:

http://bit.ly/2f0QWYL

On 2016 Oct 26, Kenneth Witwer commented:

It appears to me that this report includes PCR design errors that may invalidate the findings. In the hopes that I had made a mistake or overlooked something, I consulted with two colleagues at different academic institutions who also came to the conclusion that there are consequential errors in the assay designs. I would encourage the authors and editors to double-check what I say below and take appropriate steps if the observations are correct.

To amplify a mature miRNA, the miScript universal reverse primer used in this study must be paired with a forward primer with identity to part or all of the mature miRNA sequence. A forward primer that is the reverse complement of the mature miRNA would not amplify a specific product. However, all but one of the mature miRNA forward primers reported in the supplement, including the human miR-21 primers, are reverse complementary to the indicated miRNAs (or do not match known miRNAs, e.g., the putative MIR1508, MIR917, and MIR477 primers). Therefore, any signal obtained from these reactions would have been non-specific. The exception is MIR824; however, this miRNA does not appear to have contributed to the conclusions of the article, namely, that plant miRNAs are taken up into human circulation and affect gene expression.

Proof of the PCR design error is supplied by Supplementary Table 5, showing the sequences of PCR products of two reactions (MIR160 and MIR2673) that were cloned into a sequencing vector. Had the reactions amplified actual miScript cDNA, two features of the sequenced product should be evident: 1) mature miRNA sequence and reverse primer sequence would be separated by a poly(A) sequence (or poly(T), depending on the sequenced strand); 2) the mature miRNA sequence would come before (5' to) the poly(A) and reverse primer. In Supplementary Table 5, there is no intervening poly(A) or (T) sequence, and the mature miRNA sequence of both MIR160 and MIR2673 follows the reverse primer. It is thus clear that these sequences are not products of specific amplification of mature miRNA sequences, but rather the result of spurious amplification or cloning of the incorrectly stranded mature miRNA primers and the kit reverse primer.

Incidentally, other, less important PCR primer design and reporting errors are apparent in the supplement. The human ACTB primers do match the ACTB transcript, but are also not ideally specific, as they would amplify sequences on numerous human chromosomes. Also, several primers are designed to the minus genomic strand, not a transcript, and thus seem to have the forward and reverse labels switched.

It should also be noted that, even if the miRNA qPCR assays had been correctly designed, miR2673 is not specific to Brassica or plants, and matches low complexity sequences in organisms from human to yeast. miRBase indexes MIR2673 sequences of Medicago trunculata derived from hairpins designated MIR2673 and MIR2673a that are transcribed from chromosomes 3 and 5. The 22-nt mature sequence for both is CCUCUUCCUCUUCCUCUUCCAC, a low-complexity sequence beginning with three repeats of "CCUCUU". MIR2673 has previously been reported in pineapple (Yusuf NH, 2015), potato (Yang J, 2013), and cucumber (Wen CL, 2016). Additionally, at least one 100% match to the mature MIR2673 sequence is found on every human chromosome...along with many human transcriptome matches of 100% identity for stretches of 20 of 22 consecutive bases. To complicate matters, the curated miRBase miR2673 sequences are not used; instead, the report relies on two predicted 21-nt mature miRNA sequences at the "miRNEST 2.0" site, a site that emphasizes that neither of these putative miRNAs is supported by miRBase.

Quite possibly, transfecting massively non-physiologic amounts of plant miRNA mimics into human cells, as done in Figure 3 of this study and in another cited study (Chin AR, 2016), will elicit effects. However, these effects should not be taken as evidence of physiologic function of xenomiRs, which, assuming they are not contaminants (Tosar JP, 2014, Witwer KW, 2015), appear to reach only subhormonal (zeptomolar to attomolar) levels in human (Witwer KW, 2016).

In sum, I would conclude from these observations that no plant mature miRNA sequences were amplified from human blood, and that there was therefore no basis for the nonphysiologic transfection or gene expression studies.

On 2016 Sep 19, Josh Bittker commented:

The compound names of the format BRD#### can now be searched directly in Pubchem.

On 2016 Sep 13, Josh Bittker commented:

@Christopher Southan- Thanks, we're going to try to resolve the PubChem links by adding aliases in Pubchem of the short names used in the paper (BRD####); the compounds are registered in Pubchem with their full IDs but not the shortened IDs.

On 2016 Sep 10, Christopher Southan commented:

This post resolves the PubChem links https://cdsouthan.blogspot.se/2016/09/structures-from-latest-antimalarial.html

On 2016 Sep 16, Jonathan Eisen commented:

The first author Erica Hartmann wrote a blog post about this work at microBEnet. See http://microbe.net/2016/09/13/antimicrobial-chemicals-and-antibiotic-resistance-in-dust-now-open-access/.

On 2016 Sep 09, Cristiane N Soares commented:

The question regarding CHIK tests mentioned by Thomas Jeanne is really relevant in this case. In fact, we were concerned about co-infections, and after the paper acceptance we performed IgM and IgG CHIK tests in serum and CSF. All samples were negatives for CHIKV.

On 2016 Sep 08, Thomas Jeanne commented:

In their case report, Soares et al. do not mention testing for chikungunya virus (CHIKV), which has considerable overlap with Zika virus (ZIKV) in both epidemiologic characteristics and clinical presentation. Brazil experienced a large increase in chikungunya cases in early 2016 (Collucci C, 2016), around the time of this patient's illness, and recent case series in Ecuador (Zambrano H, 2016) and Brazil (Sardi SI, 2016) have demonstrated coinfection with ZIKV and CHIKV. Moreover, a recently published study of Nicaraguan patients found that 27% of those who tested positive for any of ZIKV, CHIKV, or DENV (dengue virus) with multplex RT-PCR also tested positive for one or both of the other viruses (Waggoner JJ, 2016). CHIKV itself has previously been linked to encephalitis including fatal encephalitis (Gérardin P, 2016), and some have speculated that adverse interactions could result from coinfection with two or more arboviruses (Singer M, 2017). Coinfection with chikungunya as a contributing factor in this case cannot be ruled out without appropriate testing.

On 2017 Mar 07, Xiaoping Liu commented:

The program and source code can be obtained in https://github.com/xp-liu/SSN.git

On 2016 Sep 08, Clive Bates commented:

This paper does not actually address the question posed in the title: Should electronic cigarette use be covered by clean indoor air laws?

The question it does address is more like "how much discomfort do vapers say they experience when the same law is applied to them as to smokers?".

The authors do not show that this is the foundation on which a justification for applying clean indoor air laws to vaping should rest. There is no basis to assume that it is.

Addressing the question set in the title is ultimately a matter of property rights. The appropriate question is: "what is the rationale for the state to intervene using the law to override the preferred vaping policy of the owners or managers of properties?".

The authors cannot simply assume that everyone and at all times shares their preference for 'clean indoor air'. Vapers may prefer a convivial vape and a venue owner may be pleased to offer them a space to do it. Unless this is creating some material hazard to other people, why should the law stop this mutually agreed arrangement? Simply arguing that it doesn't cause that much discomfort among that many vapers isn't a rationale. If the law stops them doing what they would like to do there is a welfare or utility loss to consider.

It is likely that many places will not allow vaping - sometimes for good reasons. But consider the following cases:

1. A bar wants to have a vape night every Thursday

2. A bar wants to dedicate one room where vaping is permitted

3. In a town with three bars, one decides it will cater for vapers, two decide they will not allow vaping

4. A bar manager decides on balance that his vaping customers prefer it and his other clientele are not that bothered – he’d do better allowing it

5. A hotel wants to allow vaping in its rooms and in its bar, but not in its restaurant, spa, and lobby

6. An office workplace decides to allow vaping breaks near the coffee machine to save on wasted smoking break time and encourage smokers to quit by switching

7. A care home wants to allow an indoor vaping area to encourage its smoking elderly residents to switch during the coming winter instead of going out in the cold

8. A vape shop is trying to help people switch from smoking and wants to demo products in the shop…

9. A shelter for homeless people allows it to make its clients welcome

10. A day centre for refugees allows it instead of smoking

These are all reasonable accommodations of vaping for good reasons. But the law is much too crude to manage millions of micro-judgments of this nature. It can only justify overruling them with a blanket prohibition if it is preventing harm to bystanders or workers who are exposed to hazardous agents at a level likely to cause a material risk.

A much better role for the state is to advise owners and managers on how to make these decisions in an informed way. This is what Public Health England has done [1], and that, in my view, is a more enlightened and liberal philosophy. Further, I suspect it is more likely help to convert more smokers to vaping, giving a public health dividend too.

[1] Public Health England, Use of e-cigarettes in public places and workplaces, July 2016.

On 2017 Apr 07, Anita Bandrowski commented:

This paper is the basis of part of an example Authentication of Key Biological Resources document that we and the UCSD library has put together.

Please find it here: http://doi.org/10.6075/J0RB72JC

On 2016 Dec 20, Huang Nan commented:

This paper introduced a very peculiar observation, which states that 18% and 42% of rural and urban Beijing female, with avaerge age in the early 60s, are sunbed users (Table 1). This is highly counter-intuitive as there would be less than 1% sunbed user exists in any Chinese population. Despite this observation, the authors claimed in the text that: "Only a few individuals had a sunburn history or used sunbeds."

On 2017 Aug 08, Christopher Tench commented:

Could you possibly provide the coordinates analysed otherwise it is difficult to interpret the results.

On 2016 Nov 26, Atanas G. Atanasov commented:

Excellent work, many thanks to the authors for the great overview.

On 2018 Jan 03, Thomas Hünig commented:

Thank you for bringing this up in the Commons. Yes, it is unfortunate that somewhere in production process, the "µ" symbols were converted to "m", which sometimes happens when fonts are changed. Fortunately, the mistake becomes obvious by its sheer magnitude (1000x off), and the corresponding paper in Eur. J. Immunol. with the original, correct data is referenced. My apologies that we did not spot this mistake before publication.

On 2017 Dec 31, Mark Milton commented:

The article includes several unfortunate typos in a vital piece of information. The article states that "These encouraging findings led to the design of a new healthy volunteer trial, which started at 0.1 mg/kg, i.e. a 1000- fold lower dose than the one applied in the ill-fated trial of 2006 (Clinical trials identifier: NCT01885624). After careful monitoring of each patient, the dose was gradually increased to a maximum of 7 mg/kg, still well below what had been applied in the first HV trial." The units listed for the dose are mg/kg but should have been µg/kg. The starting dose was 0.1 µg/kg and the highest dose evaluated was 7 µg/kg (Tabares et al 2014). The dose administered in the TGN1412 FIH study was 100 µg/kg. Although this typo does not detract from the overall conclusions from the study, it is sad to see that this error was not noticed by the authors or reviewers given the near tragic circumstances of the FIH clinical trial for TGN1412.

On 2016 Dec 02, Pierre Fontana commented:

Tailored antiplatelet therapy in patients at high cardiovascular risk: don’t prematurely throw the baby out with the bathwater

The clinical impact of a strategy based on a platelet function assay to adjust antiplatelet therapy has been intensively investigated. However, large prospective interventional studies failed to demonstrate the benefit of personalizing antiplatelet therapy. One of the concerns was that the interventions were delayed and partially effective, contrary to earlier smaller trials that employed incremental clopidogrel loading doses prior to PCI Tantry US, 2013 Bonello L, 2009.

Cayla and co-workers should be commended for their efforts in the ANTARCTIC trial. Although the trial is pragmatic, important limitations may account for the neutral effect of the intervention, including an antiplatelet adjustment performed between D14 and D28 after randomization. Early personalization is also supported by data from the TRITON-TIMI38 trial where half of the ischemic events (4.7/9.9%) of the prasugrel-treated arm occurred 3 days after randomization. Stratifying the analysis on the timing of events before and after D28 may provide some insight, though underpowered for a definitive conclusion.

The prognostic value of the platelet function assay and cut-off used would also be of great interest in the control group. If, the assay and cut-off values were not prognostic in this elderly population, personalization would be bound to fail.

Finally, the results of ANTARCTIC restricted to the subgroup of patients with hypertension (73% of patients), thus accumulating 3 of the risk factors related to the clinical relevance of high platelet reactivity Reny JL, 2016 would also be very interesting. Further research should not only evaluate other pharmacological approaches but also early personalization and measurement of platelet reactivity in the control group.

J.-L. Reny, MD, PhD and P. Fontana, MD, PhD

On 2016 Dec 16, Leon van Kempen commented:

RNA in FFPE tissue is commonly degraded. NanoString profiling will still yield reliable results when RNA is degraded to 200nt fragments.

On 2016 Sep 03, Adrian Barnett commented:

I should have cited this paper which shows how random funding can de-centralise funding away from ingrained ideas and hence increase overall efficiency: Sharar Avin "Funding Science by Lottery", volume 1, European Studies in Philosophy of Science

On 2016 Sep 19, G L Francis commented:

I have read your publication in PNAS titled ‘Metabolic features of chronic fatigue syndrome’ with much interest, this significant contribution has at last provided a definitive publication of a realistic evidence based diagnostic test based on a panel of blood metabolites - this could provide a more robust diagnostic base for future rational treatment studies in ‘CFS’.

Athough there are many more complex and critical questions to be asked, I will keep mine simple. I took particular note of the authors comments “When MTHFD2L is turned down in differentiated cells, less mitochondrial formate is produced and one-carbon units are directed through Methylene-THF toward increased SAM synthesis and increased DNA methylation” (from Figure S6. Mitochondrial Control of Redox, NADPH, Nucleotide, and Methylation Pathways legend). I recently read the paper, 'Association of Vitamin B12 Deficiency with Homozygosity of the TT MTHFR C677T Genotype, Hyperhomocysteinemia, and Endothelial Cell Dysfunction' Shiran A et al. IMAJ 2015; 17: 288–292, and wondered whether the gene variations in the individuals described within that publication, could be over represented in your subjects, mind you the size of your study population probably answers my own question; and no doubt many mechanisms that lead to a perturbation of this pathway exist, of which this could conceivable be just one of many, even if a minor contributor. Moreover, there does seem to be a difference between the two papers in terms of the particular pertubations on incidence of cardiovascular disease and outcomes?

On 2016 Dec 18, Laura Williams commented:

Our online journal club discussed this paper on February 22, 2016. https://www.youtube.com/watch?v=Z_tWLjjahGY

On 2016 Dec 18, Laura Williams commented:

Our online journal club discussed this paper on March 28, 2016. https://www.youtube.com/watch?v=hR96uLrta7w

On 2017 Feb 19, james brophy commented:

The authors conclude prophylactic ICD implantation "was not associated with a significantly lower long-term rate of death from any cause than was usual clinical care”. Given the observed hazard rate for death was 0.87; 95% confidence interval [CI], 0.68 to 1.12; P=0.28), this conclusion is quite simply wrong, unless a potential 32% reduction in death is considered clinically unimportant. The aphorism "Absence of proof is not proof of absence" is worth recalling.

On 2016 Aug 29, David Keller commented:

If the investigators were not blinded, how was bias excluded from their scoring of the balance, stability and TUG tests?

The placebo, nocebo, Pygmalion and other expectation effects can be substantial in Parkinson's disease. Unblinded investigators can transmit cues regarding their own expectations to patients, thereby affecting the patients' response to therapy. In addition, unblinded investigators are affected by bias in their subjective evaluations of patient response to therapy, and even in their measurement of patient performance on relatively objective tests. What was done to minimize these sources of bias from contaminating the results of this single-blinded study? Were the clinicians who scored the BBS, TUG and LOS tests aware of the randomization status of each patient they tested?

In addition, I question whether the results reported for the LOS test in the Results section of the abstract are statistically significant. The patients assigned to exergaming scored 78.9 +/- 7.65 %, which corresponds to a Confidence Interval of [71.25 - 86.55] %, while the control patient scores of 70.6 +/- 9.37 % correspond to a confidence interval of [61.23 - 79.97] %. These two confidence intervals overlap from 71.25 % to 79.97 %, a range which includes the average score of the exergaming patients.

If a follow-up study is planned, the blinding of investigators, especially those who score the patients' test performances, would reduce bias and expectation effects. Increasing the number of subjects assigned to active treatment and to control treatment would improve the statistical significance of the results.

On 2017 Jan 13, Craig Brown commented:

I would welcome more research in this matter, also. As a clinician, for many years I have found this to yield significant visual and cognitive functional improvement in many patients with mild to moderate cerebral atrophy and vascular dementia, particularly those with MTHFR polymorphisms.

Over several years, the benefit has been reliable. Patients who quit it, often come back and and restart it, because they notice loss of functional improvements that returns upon resumption.

Because Folic acid does not cross the Blood Brain Barrier, it can build up blocking active l-methylfolate transport into the brain and retina, also impairing DHFR, Dihydrofolate Reductase which impairs methylation and BH4 recycling- essential for serotonin, dopamine, and norepinephrine production- which in turn are essential for mood, attention,sleep and memory.

I find it works optimally when combined with Folic Acid avoidance- to reduce BBB blockade, riboflavin to enhance MTHFR methylation, and Vitamin D to enhance folate absorption. It has a long record of safety, with few serious side effects, for a condition that has few effective treatments. All this is to say, more research is surely a good thing here, but excessive skepticism deprives patients of a chance to try a low risk frequently helpful but not magic option.

It is classified as a Medical Food, in part, because our FDA does not encourage formulating drug products that have multiple active ingredients, particularly ingredients that occur naturally in foods and in human metabolism. Medical Foods were implemented by the FDA specifically for higher concentrations of natural food based substances important to address genetic metabolic impairments, in this case, impairment of DFHR and MTHFR; which may contribute to cerebral ischemia, atrophy, and dementia.

A final thought, double blind experiments are the ideal gold standard, however, the elderly, and the demented are considered high risk populations and have such strong protections in place at the NIH, that placebo studies are difficult to justify to, or get approval from, any Institutional Review Board IRB, when previous benefit has been shown, because that amounts to knowingly withholding treatment. We may have to content ourselves with non-placebo trials.

On 2016 Nov 09, Gayle Scott commented:

This company (Nestle Health Science - Pamlab, Inc)-sponsored study was neither randomized nor blinded. Tests of cognitive function and QOL were not included.The study will certainly be cited in advertising for CerefolinNAC.

It is important to note CerefolinNAC is a "medical food," an FDA designation for products designed to meet the nutritional needs of patients whose needs cannot be met through foods, such as inborn errors of metabolism, eg PKU (patients must avoid phenylalanine), maple syrup urine disease (patients must avoid branched chain amino acids).

Another medical food for Alzheimer's disease is Axona, caprylic triglyceride, a medium chain triglyceride found in coconut oil. Unlike dietary supplements, medical foods can be labeled for medical conditions such as Alzheimer’s disease. Dietary supplements must be labeled for so-called “structure and function claims” and cannot make claims to treat or prevent disease. For example, ginkgo may be labeled “supports memory function,” but not “for treatment of dementia.” A drug or medical food could be labeled “for treatment of dementia associated with Alzheimer’s disease.”

Think of medical foods as hybrids of prescription drugs and dietary supplements, more closely resembling dietary supplements in terms of regulation. Packaging for medical foods is similar to prescription products with package inserts, NDC numbers, and usually “Rx only” on the labels. But like dietary supplements, medical foods are not required to be evaluated for safety or efficacy, and the FDA does not require approval before marketing. "Caution: Federal law prohibits dispensing without prescription" is not required on product labeling. The FDA specifies only that these products are for use with medical supervision;. however, a medical food manufacturer may market a product to be dispensed only on physician request.

Message to patients regarding CerefolinNAC: much more research is needed.

On 2017 May 16, Michael Stillman commented:

I read this article with great interest. And with significant concern.

A sweeping review by the Department of Health and Human Services' Office for Human Research Protections of Dr. Harkema's spinal cord injury research program (https://www.hhs.gov/ohrp/compliance-and-reporting/determination-letters/2016/october-17-2016-university-louisville/index.html accessed May 16, 2017) documented numerous instances of sloppy methodologies and potential frank scientific misconduct. This report included evidence of: a) missing source documents, leading to an inability to verify whether protocols had been followed or captured data was valid; b) multiple instances of unapproved deviations from experiments protocols; c) participants having been injured while participating in translational research experiments; d) a failure to document and adjudicate adverse events and to report unanticipated problems to the IRB; and e) subjects being misled about the cost of participating in research protocols. Dr. Harkema's conduct was so concerning that the National Institute of Disability, Independent Living, and Rehabilitation Research (NIDILRR) prematurely halted and defunded one of her major research projects (http://kycir.org/2016/07/11/top-u-of-l-researcher-loses-federal-funding-for-paralysis-study/ accessed May 26, 2017).

I approached the editors of "Journal of Neurotrauma" with reports from both Health and Human Services (above) and University of Louisville's IRB and asked them three questions: a) were they adequately concerned with this study's integrity to consider a retraction; b) were they adequately concerned to consider publishing a "concerned" letter to the editor questioning the study's integrity and reliability; and c) were they interested in reviewing adverse events associated with the experiments. Their response: "no," "no," and "no."

I call on the editorial board of "Journal of Neurotrauma" to carefully inspect all documents and data sets related to this work. I would further expect them to review all adverse events reports, and to demand evidence that they've been reviewed and adjudicated by an independent medical monitor or study physician. Short of this, this work remains specious.

On 2016 Nov 03, Mohammad Al-Daydamony commented:

Excellent job my dear colleague. Keep going.

On 2016 Sep 07, Donald Forsdyke commented:

PATERNITY OF INNATE IMMUNITY?

The accolades cast on scientists we admire include that of paternity. Few will dispute that Gregor Mendel was the father of the science we now call genetics. At the outset, this paper (1) hails Metchnikoff (1845-1916) as “the father of innate immunity.” However, an obituary of US immunologist Charles Janeway (1943-2003) hails him similarly (2). Can a science have two fathers? Well, yes. But not if an alternative of Mendelian stature is around. While paternity is not directly ascribed, a review of the pioneering studies on innate immunity of Almroth Wright (1861-1947) will perhaps suggest to some that he is more deserving of that accolade (3).

1.Gordon S (2016) Phagocytosis: the legacy of Metchnikoff. Cell 166:1065-1068 Gordon S, 2016

2.Oransky I (2003) Charles A Janeway Jr. Lancet 362:409.

3.Forsdyke DR (2016) Almroth Wright, opsonins, innate immunity and the lectin pathway of complement activation: a historical perspective. Microbes & Infection 18:450-459. Forsdyke DR, 2016

On 2016 Aug 27, Martine Crasnier-Mednansky commented:

Thanks, the movies are a must-see!

On 2017 Jun 10, Shawn McGlynn commented:

With the trees from this phylogeny paper now available, we can resolve the discussion between myself and the authors (below) and conclude that there is no evidence that nitrogenase was present in the LUCA as the authors claimed in their publication.

In their data set, the authors identified two clusters of proteins which they refer to as NifD; clusters 3058 and 3899. NifD binds the metal cluster of nitrogenase and is required for catalysis. In the author's protein groups, cluster 3058 is comprised of 30 sequences, and 3899 is comprised of 10 sequences. Inspection of these sequences reveals that neither cluster contains any actual NifD sequences. This can be said with certainty since biochemistry has demonstrated that the metal cofactor coordinating residues Cys²⁷⁵ and His⁴⁴² (using the numbering scheme from the Azotobacter vinelandii NifD sequence) are absolutely required for activity. NONE of the 40 sequences analyzed by the authors contain these residues. Therefore, NONE of these sequences can have the capability to bind the nitrogenase metal cluster, and it follows that none of them would have the capacity to reduce di-nitrogen. The authors have not analyzed a single nitrogenase sequence in their analysis and are therefore disqualified from making claims about the evolution of the protein; the claims made in this paper about nitrogenase cannot be substantiated with the data which have been analyzed. The sequences contained in the author's "NifD" protein clusters are closely related homologs related to nitrogenase cofactor biosynthesis and are within a large family of related proteins (which includes real NifD proteins, but also proteins involved in bacteriochlorophyll and Ni porphyrin F430 biosynthesis). While the author's analyzed proteins are more related to nitrogen metabolism than F430 or bacteriochlorophyll biosynthesis, they are not nitrogenase, but are nitrogenase homologs that complete assembly reactions.

Other than not having looked at any sequences which would be capable of catalyzing nitrogen reduction, the presentation of two "NifD" clusters highlights important problems with the methods used in this paper which affect the entire analysis and conclusions. First, two clusters were formed for one homologous group, which should not have occurred if the goal was to investigate ancestry. Second, by selecting small clusters from whole trees, the authors were able to prune the full tree until they recovered small sub trees which show monophyly of archaea and bacteria. However it was incorrect to ignore the entire tree of homologs and present only two small clusters from a large family. This is "cherry" picking to the extreme - in this case it is "nitrogenase" picking, but it is very likely that this problem of pruning until the desired result sullies many if not all of the protein families and conclusions in the paper; for example the radical SAM tree was likely pruned in this same way with the incorrect conclusion being reached (like nitrogenase, a full tree of radical SAM does not recover the archaea bacteria split in protein phylogenies either). Until someone does a complete analysis with full trees the claims of this paper will remain unproven and misleading since they are based on selective sampling of information. It would seem that the authors have missed the full trees whilst being lost in mere branches of their phylogenetic forest of 286,514 protein clusters.

In a forthcoming publication, I will discuss in detail the branching position of the NifD homologs identified by the authors, as well as the possible evolutionary trajectory of the whole protein family with respect to the evolution of life and the nitrogen cycle on this planet in more detail, including bona fide NifD proteins which I have already made comment on below in this PubMed Commons thread.

On 2017 Mar 20, Madeline C Weiss commented:

The trees and also the alignments for all 355 proteins are available on our resources website:

http://www.molevol.de/resources/index.html?id=007weiss/

On 2016 Dec 25, Shawn McGlynn commented:

Unfortunately, the points raised by Professor Martin do not address the problem I raised in my original comment, which I quote from below: "nitrogenase protein phylogeny does not recover the monophyly of the archaea and bacteria." As I wrote, the nitrogenase protein is an excellent example of violating the author's criterion of judging a protein to be present in the LUCA by virtue that "its tree should recover bacterial and archaeal monophyly" (quoted from Weiss et alia). Therefore it should not be included in this paper's conclusions.

Let's be more specific about this and look at a phylogenetic tree of the nitrogenase D peptide (sometimes referred to as the alpha subunit). This peptide binds the catalytic metal-sulfur cluster and its phylogeny can be viewed on my google site https://sites.google.com/site/simplyshawn/home.

I colored archaeal versions red and bacterial black. You can see that this tree does not recover the monophyly of the archaea and bacteria and therefore should not be included in the author's set of LUCA proteins.

Is what I display the result of a tree construction error? Probably not, this tree looks pretty much the same to every other tree published by various methods, so it seems to correctly reflect sequence evolution as we understand it today. The tree I made just has more sequences; it can be compared directly with Figure 1 in Leigh 2000, Figure 2 in Raymond et alia 2004, and Figure 2 in Boyd et alia 2011. Unfortunately, Weiss and others do not include any trees in their paper, so it is impossible to know what they are deriving their conclusions from, but it would be very difficult to imagine that they have constructed a tree different from all of these.

Could it be that all these archaea obtained nitrogenase by horizontal gene transfer after the enzyme emerging in bacteria? Possibly, although this would imply that it was not in the LUCA as the authors claim.

Could it be that the protein developed in methanogens and was then transferred into the bacterial domain? Yes, and Boyd and others suggested just this in their 2011 Geobiology paper. This would also mean that the protein was not in the LUCA.

Could it be that the protein was present in the LUCA as Weiss and co-authors assert? Based on phylogenetic analysis, no.

As Prof. Martin writes - there certainly is more debate to be had about nitrogenase age than was visible in my first comment. However, we can be sure that the protein does not recover the archaea bacteria monophyly, and should have not been included in the authors paper.

Prof. Martin might likely counter my arguments here by saying something about metal dependence and treating different sequences separately (for example Anf, Vnf, MoFe type). However let us remember that the sequences are all homologous. Metal binding is one component of the nitrogenase phenotype, but all nitrogenase are homologous and descend from a common ancestor.

Now that we can be sure that the nitrogenase does not conform to the author's second criterion for judging presence in the LUCA, let us examine if the protein conforms to the first criterion: "the protein should be present in at least two higher taxa of bacteria and archaea". In fact, all nitrogenase in archaea that are found in the NCBI and JGI databases are only within the methanogenic euryarchaeota. Unfortunately, Weiss and coauthors do not define what "higher taxa" means to them in their article, but it should be questioned if having a gene represented by members of a single phylum actually constitutes being present within "two higher taxa". Archaea are significantly more diverse than what is observed in the methanogenic euryarchaeota. Surely, if a protein was present in the LUCA, it would be a bit more widely distributed, and it would be easy to argue that the presence of nitrogenase in only one phylum provides evidence that it does not conform to the authors criterion number one. Thus, the picture that emerges from a closer look at nitrogenase phylogeny and distribution is that the protein violates both of the authors criteria for inclusion in the LUCA protein set.

Let me summarize:

1) Nitrogenase does not recover the bacterial and archaeal monophyly and therefore violates the author's criterion number 2.

2) Nitrogenase in archaea is only found within the methanogenic euryarchaeota and is not broadly distributed, and therefore also seems to violate the authors criterion number 1.

3) From a phylogenetic perspective, the nitrogenase protein should not be included as a candidate to be present in the LUCA.

On 2016 Oct 12, William F Martin commented:

There is an ongoing debate in the literature about the age of nitrogenase.

In his comment, McGlynn favours published interpretations that molybdenum nitrogenase arose some time after the Great Oxidation Event 2.5 billion years ago (1). A different perspective on the issue is provided by Stüecken et al. (2) who found evidence for Mo-nitrogenase before 3.2 billion years ago. Our recent paper (3) traced nitrogenase to LUCA, but also suggested that methanogens are the ancestral forms of archaea, in line both with phylogenetic (4) and isotope (5) evidence for the antiquity of methanogens, and with a methanogen origin of nitrogenase (6).

Clearly, there had to be a source of reduced nitrogen at life’s origin before the origin of nitrogenase or any other enzyme. Our data (3) are consistent with the view that life arose in hydrothermal vents and independent laboratory studies show that dinitrogen can be reduced to ammonium under simulated vent conditions (7,8). There is more to the debate about nitrogenase age, methanogen age, and early sources of fixed nitrogen than McGlynn’s comment would suggest.

1. Boyd, E. S., Hamilton, T. L., and Peters, J. W. (2011). An alternative path for the evolution of biological nitrogen fixation. Front. Microbiol. 2:205. doi:10.3389/fmicb.2011.00205

2. Stüeken EE, Buick R, Guy BM, Koehler MC. Isotopic evidence for biological nitrogen fixation by molybdenum-nitrogenase from 3.2 Gyr. Nature 520, 666–669 (2015)

3. Weiss MC, Sousa FL, Mrnjavac N, Neukirchen S, Roettger M, Nelson-Sathi S, Martin WF: The physiology and habitat of the last universal common ancestor. Nat Microbiol (2016) 1(9):16116 doi:10.1038/nmicrobiol.2016.116

4. Raymann, K., Brochier-Armanet, C. & Gribaldo, S. The two-domain tree of life is linked to a new root for the Archaea. Proc. Natl Acad. Sci. USA 112, 6670–6675 (2015).

5. Ueno, Y., K. Yamada, N. Yoshida, S. Maruyama, and Y. Isozaki. 2006. Evidence from fluid inclusions for microbial methanogenesis in the early archaean era. Nature 440:516-519.

6. Boyd, E. S., Anbar, A. D., Miller, S., Hamilton, T. L., Lavin, M., and Peters, J. W. (2011). A late methanogen origin for molybdenum-depen- dent nitrogenase. Geobiology 9, 221–232.

7. Smirnov A, Hausner D, Laffers R, Strongin DR, Schoonen MAA. Abiotic ammonium formation in the presence of Ni-Fe metals and alloys and its implications for the Hadean nitrogen cycle. Geochemical Transactions 9:5 (2008) doi:10.1186/1467-4866-9-5

8. Dörr M, Kassbohrer J, Grunert R, Kreisel G, Brand WA, Werner RA, Geilmann H, Apfel C, Robl C, Weigand W: A possible prebiotic formation of ammonia from dinitrogen on iron sulfide surfaces. Angew Chem Int Ed Engl 2003, 42(13):1540-1543.

On 2017 Mar 09, Tanai Cardona commented:

I agree with Shawn regarding the fact that "Nitrogenase does not recover the bacterial and archaeal monophyly and therefore violates the author's criterion number 2."

I have a different explanation for why nitrogenase was recovered in LUCA. And this has to do with the tetrapyrrole biosynthesis enzymes related to nitrogenases that, in fact, do recover monophyly for Archaea and Bacteria. Namely, the enzyme involved in the synthesis of the Ni-tetrapyrrole cofactor, Cofactor F430, required for methanogenesis in archaea; and the enzymes involved in the synthesis of Mg-tetrapyrroles in photosynthetic bacteria. Still to this date, the subunits of the nitrogenase-like enzyme required for Cofactor F430 synthesis are annotated as nitrogenase subunits.

So, what Weiss et al interpreted as a nitrogenase in LUCA, might actually include proteins of the tetrapyrrole biosynthesis enzymes.

Bill, I think you should make all the trees for each one of the 355 proteins available online. That would be really useful for all of us interested in early evolution! Thank you.

On 2016 Oct 08, Shawn McGlynn commented:

This paper uses a phylogenetic approach to "illuminate the biology of LUCA" and uses two criteria to assess if a given protein encoding gene was in the LUCA:

"the protein should be present in at least two higher taxa of bacteria and archaea, respectively, and (2) its tree should recover bacterial and archaeal monophyly"

The authors later conclude that "LUCA accessed nitrogen via nitrogenase", however the nitrogenase protein is an excellent example of violating the author's criterion (2) above, and therefore cannot be included in the LUCA protein set based on the author's own criterion.

Upon phylogenetic analysis, the nitrogenase alpha subunit protein - which ligates the active site - branches into five clusters. One of these clusters is not well resolved, yet four of the five clusters contain both archaea and bacteria, therefor a nitrogenase protein phylogeny does not recover the monophyly of the archaea and bacteria.

Other claims in this paper may deserve scrutiny as well.

Suggested Reading below - if there are others to add someone please feel free:

Raymond, J., Siefert, J. L., Staples, C. R., and Blankenship, R. E. (2004). The natural history of nitrogen fixation. Mol. Biol. Evol. 21, 541–554

Boyd, E. S., Anbar, A. D., Miller, S., Hamilton, T. L., Lavin, M., and Peters, J. W. (2011a). A late methanogen origin for molybdenum-depen- dent nitrogenase. Geobiology 9, 221–232.

Boyd, E. S., Hamilton, T. L., and Peters, J. W. (2011b). An alternative path for the evolution of biological nitrogen fixation. Front. Microbiol. 2:205. doi: 10.3389/fmicb.2011.00205

On 2016 Oct 19, DP Zhou commented:

China's fragile health insurance system can not serve the health system. Patients are forced to spend all their savings to buy medicines in cash with no insurance coverage. For most families, one cancer patient means the bankruptcy of the whole family. In such despair, many patients choose to extort the doctors and the hospitals as the last option to recover some cost of medicines.

The health insurance system in China is a sensitive issue. The state-provided insurance is not covering major illnesses. The private insurance is poor-quality and mostly abused by finance institutions in real estate investment and other speculations.

On 2016 Aug 30, Peter Hajek commented:

The length of exposure would be relevant if the dosing was comparable, but the damage to mice lungs was caused by doses of nicotine that were many times above anything a human vaper could possibly get. It is the dose that makes the poison. Many chemicals produce damage at large enough doses while lifetime exposure to small enough doses is innocent.

To justify the conclusions about toxicity of vaping, the toxic effect would need to be documented with realistic dosing, and then shown to actually apply to humans (who have much better nicotine tolerance than mice).

I agree that mice studies with realistic dosing could be useful, though data on changes in lung function in human vapers would be much more informative; and I do appreciate that the warnings of risks in the paper were phrased with caution.

On 2016 Aug 30, Robert Foronjy commented:

The study is NOT reassuring to e-cigarette consumers. On the contrary, it shows that nicotine exposure reproduced the lung structural and physiologic changes present in COPD. These changes occurred after only four months of exposure. Even adjusting for the differences in lifespans, this exposure in mice is much briefer than that of a lifelong e-cigarette consumer. I do agree, however, that carefully conducted studies are needed to determine whether there is a threshold effect of nicotine exposure.

On 2016 Aug 29, Peter Hajek commented:

Thank you for the explanation. The exposure however was not equivalent. Mice have much faster nicotine metabolism than humans which means that nicotine exposure in mice must be many times higher than in humans to produce the same blood cotinine levels. See the reference below that calculated that mice with comparable cotinine levels were exposed to an equivalent of at least 200 cigarettes per day. In addition to this, mice also have much lower tolerance to nicotine than humans which means that their organs would be much more severely affected even if the levels were comparable.

On 2016 Aug 29, Robert Foronjy commented:

Mortality was not reported in the manuscript since no deaths occurred. The exposure was well tolerated by the mice and no abnormal behavior or physiologic stress was noted. At the time of euthanasia, all the internal organs were grossly normal on exam. Cotinine levels in the mice were provided in the study and they are similar to what has been documented in humans who vape electronic cigarettes. We agree that both the mice and human consumers are exposing their lungs to toxic concentrations of nicotine. This is one of the essential points that is expressed by the data presented in the manuscript.

On 2016 Aug 26, Peter Hajek commented:

The authors propose a hypothesis that deserves attention, but the study findings need to be interpreted with caution.

The mice were severely overdosed with nicotine, up to the lethal levels for mice, and a huge amount above what any human vaper would get - see this comment on a previous such study:

http://journals.plos.org/plosone/article/comment?id=info:doi/10.1371/annotation/5dfe1e98-3100-4102-a425-a647b9459456

The report does not say how many mice were involved and if any died during the experiment; and whether effects of nicotine poisoning were detected in other organ systems. This could perhaps be clarified.

Regarding the relevance to human health, nicotine poisoning poses normally no risk to vapers or smokers because if nicotine concentrations start to rise above their usual moderate levels, there is an advance warning in the form of nausea which makes people stop nicotine intake long before any dangerous levels can accrue. (Mice in these types of experiments do not have that option).

The study actually provides a reassurance for vapers, to the extent that mice outcomes have any relevance for humans, in that in the absence of nicotine overdose, chronic dosing with the standard ingredients of e-cigarette aerosol (PG and VG) had no adverse effects on mice lungs.

Peter Hajek

On 2016 Sep 17, Jonathan Eisen commented:

Katehine Dahlhausen wrote a brief blog post about this paper at microBEnet. See http://microbe.net/2016/08/30/journal-club-factors-influencing-early-life-gut-microbial-communities/.

On 2016 Sep 04, Egon Willighagen commented: