On 2017 Apr 01, Misha Koksharov commented:

It's interesting whether LREs have a gluconolactonase function. Or whether gluconolactonases have the LRE activity

On 2017 Mar 26, Misha Koksharov commented:

None

On 2017 Jul 04, Randi Pechacek commented:

Matt Perisin, first author of this paper, wrote a "Behind the Paper" blog on microBEnet.

On 2016 Feb 16, Robert Courtney commented:

A study that uses questionable assumptions rather than empirical evidence leads to conclusions that stretch credibility.

Chalder et al. [1] used the “single mediation model” for their methodology, which is explained in detail in a book by MacKinnon [2]. Explaining the methodology MacKinnon says a temporal separation between variables must be observed (i.e. changes in mediating variable must occur before changes in the mediated variable) for a mediation effect to be empirically and robustly established.

Chalder et al. were working to this model and acknowledged that they failed to establish a clear temporal separation between variables, and therefore did not empirically establish a causal mediation effect: “Given the pattern of change in the mediators was similar to the pattern of change in the outcomes it is possible that the variables were affecting each other reciprocally”.

However, despite the lack of robust empirical evidence to support a mediation effect, the investigators concluded that they had established mediation effects, e.g: “Our main finding was that fear avoidance beliefs were the strongest mediator for both CBT and GET.”

The study’s conclusion relied upon an assumption that the investigators’ favoured hypothetical model of illness for ME/CFS has a robust empirical evidence base and is applicable to this study. The hypothesis is based upon the idea that symptoms and disability in ME/CFS are perpetuated by unhelpful or maladaptive illness beliefs, fear, and an avoidance of activity.

However, the prestigious National Academy of Medicine (formerly known as the Institute of Medicine) recently released a comprehensive report [3] into ME/CFS that rejected such a hypothetical model of illness, and unambiguously concluded that ME/CFS does not have a psychological or cognitive-behavioural basis, but is an organic illness that requires biomedical research.

Chalder et al. discussed the possibility that more frequent measurements may have potentially demonstrated a temporal separation between the variables, and therefore a mediation effect. However, this raises the possibility of whether changes in the primary outcome variables (self-report physical function and fatigue) may, in fact, have occurred before changes in the presumed mediator variables. Such an outcome would entirely contradict the investigators’ premature conclusions. According to MacKinnon [2] and Wiedermann et al. [4], unexpected outcomes should not be ruled out.

Chalder et al. concluded that symptoms and physical impairment, in ME/CFS patients, are mediated by activity avoidance and other factors. (e.g. This would mean that a decrease in activity would cause an increase in symptoms.) However, from a common sense point of view, this seems like rather a convoluted conclusion, and it seems more likely that increased symptoms would be the direct cause of activity avoidance in any illness, rather than vice versa. To conclude that activity avoidance causes fatigue (rather than fatigue being a direct cause of activity avoidance), is similar to concluding that a person has flu because they’ve taken a day off work, rather than the obvious conclusion that they’ve taken a day off work because they have flu.

In the case of fatigue, flu-like malaise and other symptoms of ME/CFS, it seems reasonable to consider the possibility that, as the symptoms fluctuate, patients may intuitively or rationally adapt their activity levels according to what is comfortable and safe. i.e. patients reduce activity levels because they are fatigued. The investigators have concluded that patients are fatigued because they have reduced activity levels.

Perhaps patients’ perspectives and insights would help clarify the issues but, unfortunately, patients were not consulted for this study.

References:

1. Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 2015; 2: 141–52.

2. MacKinnon DP. Introduction to Statistical Mediation Analysis. Taylor and Francis: New York 2008.

3. IOM (Institute of Medicine). 2015. Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Washington, DC: The National Academies Press. http://iom.nationalacademies.org/Reports/2015/ME-CFS.aspx

4. Wiedermann W, von Eye A. Direction of Effects in Mediation Analysis. Psychol Methods 2015; 20: 221-44.

On 2015 Sep 15, Tom Kindlon commented:

(Contd.)

References:

1 Torjesen I. Tackling fears about exercise is important for ME treatment, analysis indicates. BMJ 2015;350:h227 http://www.bmj.com/content/350/bmj.h227

2 Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 14 Jan 2015, doi:10.1016/S2215-0366(14)00069-8.

3 Burgess M, Chalder T. Manual for Participants. Cognitive behaviour therapy for CFS/ME.http://www.pacetrial.org/docs/cbt-participant-manual.pdf (accessed: January 17, 2015)

4 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME. Information for Participants http://www.pacetrial.org/docs/get-participant-manual.pdf (accessed: January 17, 2015)

5 Wechsler ME, Kelley JM, Boyd IO, Dutile S, Marigowda G, Kirsch I, Israel E, Kaptchuk TJ. Active albuterol or placebo, sham acupuncture, or no intervention in asthma. N Engl J Med. 2011;365(2):119-26.

6 Whiting P, Bagnall AM, Sowden AJ, Cornell JE, Mulrow CD, Ramírez G. Interventions for the treatment and management of chronic fatigue syndrome: a systematic review. JAMA. 2001 Sep 19;286(11):1360-8.

7 Burgess M, Chalder T. PACE manual for therapists. Cognitive behaviour therapy for CFS/ME.http://www.pacetrial.org/docs/cbt-therapist-manual.pdf (accessed: January 17, 2015)

8 White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, et al, for the PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011;377:823-36.

9 McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS One. 2012;7(8):e40808. doi: 10.1371/journal.pone.0040808

10 Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg G. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychol Med. 2010 Aug;40(8):1281-7. doi: 10.1017/S0033291709992212. Epub 2010 Jan 5.

11 Heins MJ, Knoop H, Burk WJ, Bleijenberg G. The process of cognitive behaviour therapy for chronic fatigue syndrome: which changes in perpetuating cognitions and behaviour are related to a reduction in fatigue? J Psychosom Res. 2013 Sep;75(3):235-41. doi: 10.1016/j.jpsychores.2013.06.034. Epub 2013 Jul 19.

12 Friedberg F, Sohl S. Cognitive-behavior therapy in chronic fatigue syndrome: is improvement related to increased physical activity? J Clin Psychol. 2009 Apr;65(4):423-42. doi: 10.1002/jclp.20551.

13 Knoop H, Prins JB, Stulemeijer M, van der Meer JW, Bleijenberg G. The effect of cognitive behaviour therapy for chronic fatigue syndrome on self-reported cognitive impairments and neuropsychological test performance. Journal of Neurology and Neurosurgery Psychiatry. 2007 Apr;78(4):434-6.

14 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME (Therapist manual): http://www.pacetrial.org/docs/get-therapist-manual.pdf (accessed: January 17, 2015)

15 O'Dowd H, Gladwell P, Rogers CA, Hollinghurst S, Gregory A. Cognitive behavioural therapy in chronic fatigue syndrome: a randomised controlled trial of an outpatient group programme. Health Technology Assessment, 2006, 10, 37, 1-140.

16 Knoop H, Wiborg JF. What makes a difference in chronic fatigue syndrome? Lancet Psychiatry 13 Jan 2015 DOI: http://dx.doi.org/10.1016/S2215-0366(14)00145-X

17 Kindlon T. Reporting of Harms Associated with Graded Exercise Therapy and Cognitive Behavioural Therapy in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Bulletin of the IACFS/ME. 2011;19(2):59-111http://iacfsme.org/BULLETINFALL2011/Fall2011KindlonHarmsPaperABSTRACT/ta...

18 Lipkin DP, Scriven AJ, Crake T, Poole-Wilson PA. Six minute walking test for assessing exercise capacity in chronic heart failure. Br Med J (Clin Res Ed) 1986. 292:653–655. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1339640/pdf/bmjcred00224-001...

19 Marin JM, Carrizo SJ, Gascon M, Sanchez A, Gallego B, Celli BR. Inspiratory Capacity, Dynamic Hyperinflation, Breathlessness, and Exercise Performance during the 6-Minute-Walk Test in Chronic Obstructive Pulmonary Disease. Am. J. Respir. Crit. Care Med. 2001 63(6):1395-1399.http://171.66.122.149/content/163/6/1395.full

20 Goldman MD, Marrie RA, Cohen JA. Evaluation of the six-minute walk in multiple sclerosis subjects and healthy controls. Multiple Sclerosis 2008. 14(3):383-390. http://pocketknowledge.tc.columbia.edu/home.php/viewfile/download/65399/The six-minute walk test.pdf

21 Ross RM, Murthy JN, Wollak ID, Jackson AS. The six minute walk test accurately estimates mean peak oxygen uptake. BMC Pulm Med. 2010 May 26;10:31. PMID 20504351.http://www.biomedcentral.com/1471-2466/10/31

22 Camarri B, Eastwood PR, Cecins NM, Thompson PJ, Jenkins S. Six minute walk distance in healthy subjects aged 55–75 years. Respir Med. 2006. 100:658-65 http://www.resmedjournal.com/article/S0954-6111(05)00326-4/abstract

23 Troosters T, Gosselink R, Decramer M. Six minute walking distance in healthy elderly subjects. Eur Respir J. 1999. 14:270-4. http://www.ersj.org.uk/content/14/2/270.full.pdf

24 Rapport d'évaluation (2002-2004) portant sur l'exécution des conventions de rééducation entre le Comité de l'assurance soins de santé (institué auprès de l'Institut national d'assurance maladie invalidité) et les Centres de référence pour le Syndrome de fatigue chronique (SFC), Bruxelles, juillet 2006. (French language edition)

25 Evaluatierapport (2002-2004) met betrekking tot de uitvoering van de revalidatieovereenkomsten tussen het Comité van de verzekering voor geneeskundige verzorging (ingesteld bij het Rijksinstituut voor Ziekte- en invaliditeitsverzekering) en de Referentiecentra voor het Chronisch vermoeidheidssyndroom (CVS). 2006. Available online:https://drive.google.com/file/d/0BxnVj9ZqRgk0QTVsU2NNLWJSblU/edit (accessed: January 17, 2015) (Dutch language version)

26 Stordeur S, Thiry N, Eyssen M. Chronisch Vermoeidheidssyndroom: diagnose, behandeling en zorgorganisatie. Health Services Research (HSR). Brussel: Federaal Kenniscentrum voor de Gezondheidszorg (KCE); 2008. KCE reports 88A (D/2008/10.273/58)https://kce.fgov.be/sites/default/files/page_documents/d20081027358.pdf (accessed: January 17, 2015)

On 2015 Sep 15, Tom Kindlon commented:

Objective measures found a lack of improvement for CBT & GET in the PACE Trial: subjective improvements may simply represent response biases or placebo effects in this non-blinded trial

[Originally posted here: http://www.bmj.com/content/350/bmj.h227/rr-10]

This BMJ article and a flurry of articles in the lay media this week followed the publication in Lancet Psychiatry of an analysis of the mediators of change in the important PACE Trial, a chronic fatigue syndrome (CFS) trial which cost UK taxpayers £5 million[1,2]. What seems to have been lost in the coverage is that, although there were some modest improvements in the self-report measures, there was an almost complete absence of improvements in objectively measured outcomes for cognitive behavioural therapy (CBT) and graded exercise therapy (GET) compared to the control group (specialist medical care only (SMC)).

This is a non-blinded trial, where participants were told CBT and GET had previously been found to be effective in CFS and other conditions[3,4]: one way to look at the mediation results for subjective measures when there was a lack of objective improvements is that they may merely tell us how response biases and/or placebo effects are mediated[5].

The focus on subjective measures in some CFS studies was previously criticised in a systematic review published back in 2001 (long before the PACE Trial started)[6]. They suggested instead "a more objective measure of the effect of any intervention would be whether participants have increased their working hours, returned to work or school, or increased their physical activities."

The model presented for cognitive behaviour therapy (CBT) in the PACE Trial manuals posits that the impairments and symptoms are reversible with the therapy[3,7]. However, the latest paper shows that fitness, as measured by a step test, didn't improve following CBT[2]. An earlier PACE Trial publication reported that the addition of CBT to SMC did not result in an improvement in 6-minute walking test scores compared to SMC alone[8].

The PACE Trial was part funded by the UK Department of Work and Pensions, a rare move for them, presumably done due to an expectation that the therapies would improve measures of employment and levels of benefit receipt. However, again CBT brought about no improvement using objective measures, such as days of employment lost, levels of disability benefits received and levels of receipt of insurance payments[9].

These results are in line with earlier studies of CBT. For example, an analysis of three randomized controlled trials of CBT interventions for CFS found no improvement in objectively measured activity, despite participants reporting a reduction in (self-reported) fatigue and (sometimes) functional impairments[10]. Similar results were found in another uncontrolled trial where changes in objectively measured activity did not predict fatigue levels, and objectively measured activity on completion remained low compared to population norms[11]. An uncontrolled study found improvements in self-reported physical functioning and fatigue were reported despite a numerical decrease in (objectively measured) activity[12]. In another study, the level of self-reported cognitive impairment in CFS patients decreased significantly after CBT, however, cognition had not improved when it was measured objectively using neuropsychological test performance[13].

It is unsurprising that 15 sessions of CBT (and the associated homework exercises and management program) might alter how participants respond to self-report questionnaires. A PACE Trial manual itself says "the essence of CBT is helping the participant to change their interpretation of symptoms": this could lead to altered or biased fatigue scores, one of the two primary outcome measures[14]. Also, one of the aims of CBT (for CFS) has been said to be "increased confidence in exercise and physical activity"[15]. The possible responses for the other primary outcome measure, the SF-36 physical functioning subscale, are "yes, limited a lot", "yes, limited a little" and "no, not limited at all" to questions on a range of physical activities. Such responses could be easily be artificially altered following a therapy like CBT for CFS.

The results were not that different with the GET cohort in the PACE Trial. Again the manuals predicted that the impairments and symptoms are reversible using the intervention[4,15]. The model said there was no reason participants should not be able to get back to full functioning. Deconditioning was posited to be an important maintaining factor. However, GET did not result in an improvement in fitness, as measured by the step test. GET did result in a small improvement on the six minute walking test to a final distance of 379 metres, or 35 metres more than the SMC-only group[7]. However, as Knoop and Wiborg commented in an accompanying editorial in Lancet Psychiatry: "an increase in distance walked during a test situation without an increased fitness suggests that patients walk more because of a change in cognitive processes (eg, daring to do more or an increased self-efficacy with respect to activity), not because of a change in physiological capacity”[16]. The result remained very poor given that normative data would suggest a group of similar age and gender should walk an average of 644 or so metres[17]. The distance walked remained comparable to people with many serious conditions[18-21], and considerably worse than the distance walked by healthy elderly adults[22,23], despite the PACE trial cohort having a mean age of only 40[8]. Also, to be allowed entry into CFS research studies such as the PACE Trial one can not have a range of chronic illnesses so with genuine recovery one would expect results comparable to healthy people[8].

As with CBT, measures relating to employment showed no improvement following GET in days of work missed, which remained very high, nor a reduction in levels of benefits (financial support from the state) or payments from insurance companies[9].

These results are in line with an audit of Belgian rehabilitation centres for CFS offering CBT and GET[24-26]. Some improvements in subjective measures were found, but there was no improvement in the results of the exercise test and hours in employment actually decreased.

Probably the main contribution of the PACE Trial has been to add to a growing body of evidence that while CBT and GET for CFS have resulted in some changes on subjective measures, they haven't lead to improvements on objective measures.

Competing interests: I am a committee member of the Irish ME/CFS Association and perform various types of voluntary work for the Association.

(continues)

On 2015 Oct 15, Radboudumc Psycho-Oncology Journal Club commented:

This interesting commentary, which has important implications for the field of psycho-oncology, was discussed by our Journal Club members on 14th of October, 2015 and generated a lively discussion and the following comments and questions:

• 1) After reading your commentary, we concluded that it would be worthwhile to incorporate lifestyle variables in psychosocial intervention programmes on tertiary prevention, to further explore the relationship between psychological outcomes and health behaviours. Our group is based within an academic medical centre and there was debate within our group about the role of psycho-oncologists should take. Some members of our journal club believed that psycho-oncology as a discipline should focus more on collaboration with public health, health psychology and other disciplines working in the area of primary and secondary cancer prevention.. However, other members believed health behaviour change is the area of expertise of health psychologists not psycho-oncologists. Furthermore, in light of resource constraints, psycho-oncology professionals should focus on the core business of providing care to those affected by disease and ill-health in medical settings.<br>
• 2) We welcomed the proposal of The Expanded Model of Research in Psycho-Oncology and congratulate the authors for stimulating debate over the role of psycho-oncology. As for a suggestion, we would have liked it if the new conceptional model had an additional box with factors relevant to secondary prevention after cancer diagnosis. For example, psycho-oncologists are well placed to conduct activities that focus on the secondary prevention of psychological problems (e.g. depression, anxiety) or which might prevent a new disease recurrence following a diagnosis of cancer (e.g via lifestyle change programmes for cancer survivors). We also believed that psychological screening within oncology settings can be conceptualised as a secondary prevention activity.
• 3) We believe that psycho-oncology as a discipline, if it is to be involved with primary and secondary prevention at all, might focus on the three non-tumor specific health behavior changes with the biggest impact smoking, diet and exercise.
• 4) When it comes to medical psychologists working in psycho-oncology, essentially it comes down to the question whether a hospital is a “health valley” or a place in which we treat those affected by disease. Primary and secondary prevention implies the “health valley” model, which needs a much larger paradigm shift of medicine in general rather than in psycho-oncology alone. It will also require increased funding for hospitals to enable this to occur.<br>
• 5) We identified systemic barriers such as the organisational separation of hospitals and public health programmes in many jurisdictions. We concur with the authors that further investment is needed in the training of psycho-oncologists to ensure they learn how to apply their skills to preventive health and have more opportunities to gain work experience in preventive health care settings.

On 2015 Oct 02, Robert Groom commented:

Thank you to the authors for this tribute to an outstanding perfusionist that invested herself in improving her field. The commentary from her friends and contemporaries exemplifies the importance of our relationships with colleagues in professional societies and how through those relationships we are able to make the World a better place.

On 2015 Dec 06, Gwinyai Masukume commented:

Unacknowledged important limitation - the effective sample size of 229 was insufficient to detect a statistically significant association.

Dean and colleagues find that there was a deficit of 229 pregnancies among women who were in the first trimester when the mass shooting at Port Arthur occurred and that by sex there was no statistically significant differential loss. They estimate that for every 100 females, 107 males were lost.

Although Dean and colleagues did not find a statistically significant difference by sex, their sample size of 229 was insufficient to detect the difference given the small effect size Austad SN, 2015.

A rigorous study Orzack SH, 2015 demonstrated that normally, in the absence of exogenous stressors such as that caused by the Port Arthur calamity, more female embryos/fetuses are lost during the first trimester, approximately 100 females for every 97 males.

In conclusion, the results presented by Dean and colleagues show excess male loss during a period when more female loss is expected. Thus, the small sample size of 229 should have been mentioned as an important limitation of the paper and a formal power calculation was apt.

On 2015 Sep 17, Daniel Himmelstein commented:

Thanks Dr. Levine for your thoughtful response. As you mention, the practices I criticize in your discovery phase are not unique to your study. Your study caught my attention because of its remarkable finding that such a small sample yielded a highly predictive and robust classifier for such a complex phenotype. Hopefully, others will benefit from our discussion here.

Additionally, I agree that replication grants researchers the freedom to discover as they wish. Suboptimal model training should not cause "type I" replication error, if the replication dataset is independent.

However, a replication p-value alone is insufficient to identify the probability of a model being true. This probability depends on the plausibility of the model. Since I think that the odds are low that your study design could produce a true model, I require extraordinary evidence before accepting the proposed PRS model. I think your replication provides good evidence but not extraordinary.

Follow-up studies on different populations will be important for establishing extraordinary evidence. I think it would be helpful to specify which allele is minor for each PRS SNP: my impression is that minor alleles are sample not population specific.

On 2015 Sep 16, Morgan E Levine commented:

Daniel Himmelstein, thank you for your comments. I will try to address them to the best of my ability.

We acknowledge that the sample size is very small, which we mention in our limitations section of the paper. Because we are studying such a rare phenotype, there is not much that can be done about this. “Long-lived smokers” is a phenotype that has been the subject of a number of our papers, and that we think has strong genetic underpinnings. Despite the small sample size, we decided to go ahead and see if we could detect a signal, since there is evidence to suggest that the genetic influence may be larger for this phenotype compared to many others—something we discuss at length in our introduction section.

To the best of our knowledge the finding that highly connected genes contain more SNPs, has not been published in a peer-reviewed journal. Therefore, we had no way of knowing or evaluating the importance of this for our study. Similarly, we used commonly used networks and do acknowledge the limitations of these networks in our discussion section. The network you link to was not available at the time this manuscript was accepted.

We acknowledge the likelihood of over-fitting in our PRS, which is probably due to our sample size. This score did validate in two independent samples. Therefore, while it is likely not perfect, we feel that it may still capture some of the true underlying signal. We followed standard protocol for calculating our score (which we reference). In the literature there are many examples of scores that have been generated by linearly combining information from SNPs that are below a given p-value threshold in a GWAS. While, not all of these replicate, many do. Our study used very similar methods, but just introduced one additional SNP selection criteria—SNPs had to also be in genes that were part of an FI network. I don't think this last criteria would introduce additional bias that would cause a type I error in the replication analysis. However, we still recognize and mention some of the limitations of our PRS. We make no claim that the score is free from error/noise or that it should be used in a clinical setting. In fact, in the paper we suggest future methods that can be used to generate better scores.

We feel we have provided sufficient information for replication of our study. The minor alleles we used are consistent with those reported for CEU populations, which is information that is readily available. Thus, the only information we provide in Table S2 pertain to things specific to our study, that can't be found elsewhere. Lastly, the binning of ages is not 'bizarre' from a biogerontology and longevity research perspective. A number of leaders in the filed have hypothesized that the association between genes and lifespan is not linear (variants that influence survival to age 100+ are not the same as variants that influence survival to age 80+). Thus, using a linear model would not be appropriate in this case and instead we selected to look at survival by age group.

On 2015 Sep 15, Daniel Himmelstein commented:

I have several concerns with the discovery phase of this study. Specifically:

Underpowered: The sample size (90 cases) is insufficient for a complex phenotype where effect sizes are small. My own research, Himmelstein DS, 2015, does not consider GWAS with under 1000 samples because any findings are likely to be false (Sawcer S, 2008). The pathway-based prioritization attempts to alleviate the underpowered study design but suffers from the following two criticisms.

SNP abundance confounding: Genes were selected for the network analysis if they contained any SNPs with p < 5×10^-3. Therefore, genes containing more SNPs were more likely to get selected by chance. Since genes with more SNPs appear more frequently in curated pathway databases, the enrichment analysis is confounded. A permutation test that shuffles case-control status and recomputes SNP p-values would prevent SNP abundance confounding. However, this does not appear to be the permutation test that was performed.

Limited network relevance: The network analysis uses only curated pathway databases. These databases are heavily biased towards well-studied genes as well as being incomplete. In our recent network that includes more curated pathways than Reactome FI, only 9,511 genes are in any curated pathway. In other words, the majority of genes aren't curated to a single pathway and hence cannot contribute to this study's prioritization approach.

Overfitting: The polygenic risk score (PRS) was ruthlessly overfit. The PRS perfectly discriminated the 90 long-lived smokers from the younger smokers. The authors don't appear to appreciate that the performance is due to overfitting and write:

Results showed that the score completely accounted for group membership, with no overlap between the two groups.

Not only were scores significantly higher for the long-lived group, but scores also appeared to be more homogeneous.

Egregious overfitting is guaranteed by their PRS approach since 215 logistic regressions are fit, each with only 90 positives and without regularization or cross-validation. When a model is overfit on training data, its perfomance on novel data diminishes.

Unreplicable: Table S2 of the supplement does not specify which allele is minor for each SNP. Therefore, the PRS computation cannot be replicated by others.

Given these issues, I find it unlikely that the study found a reliable genotype of longevity. Rather, I suspect the successful validation resulted from confounding, p-value selection bias, or an implementation error.

Finally, the binning of continuous outcomes, primarily age, is bizarre. The binning serves only to reduce the study's power, while providing much room for unintended p-value selection bias.

Update 2015-09-15: I am not suggesting any misconduct, negligence, or intentional bad practices. Rather the methods are clearly described and the validation quite impressive and seemingly honest. I believe the study makes a valuable contribution by proposing a genotype of longevity, which future studies can confirm or deny.

Update 2015-09-19: I replaced "p-hacking" with "p-value selection bias". My intended meaning is the greater investigation and preferential publication granted to more significant findings.

On 2017 Oct 21, Anju Anand commented:

This article was discussed at our #rsjc. This captures the summary: http://www.thelancet.com/journals/lanres/article/PIIS2213-2600(15)00523-8/fulltext

On 2016 Jun 13, Stuart Phillips commented:

Please see a letter to the editor regarding the results from this paper: http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=10353765&fileId=S000711451600221X

On 2016 Apr 30, Stuart Phillips commented:

None

On 2016 Apr 30, Stuart Phillips commented:

None

On 2015 Sep 18, Alejandro Diaz commented:

Prepubertal gynecomastia is a very rare condition and there have been an unexpected number of cases reported in the Miami area. This case report highlighted three cases. However, we have seen many other children in our practices with prepubertal gynecomastia or thelarche who were similarly exposed to the lavender-containing cologne described in our report. Unfortunately, the evaluation and follow up for these children was not as in-depth as in the cases that we presented in the article, which included chemical analysis of the cologne product.

In Politano VT, 2013 article, uterotrophic assay, performed in immature female rats, was used to evaluate estrogenic activity in vivo. However, these rats were only treated with lavender oil for a period of 3 days, whereas the children we reported on were exposed to the lavender-containing cologne for several years. As described by Henley DV, 2007, the estrogenic activity of lavender is weak, which may explain why there was not uterotrophic effects on these rats.

You mentioned that the reason Henley found that lavender and tea tree oil activated the estrogen receptor was due to the use of polystyrene in their test system as opposed to the use of glass. Both Ohno K, 2003 and Fail PA, 1998 could not demonstrate estrogenic response from polystyrene. Therefore, this explanation is not substantiated. Moreover, there was an estrogenic response to the lavender and tea tree oils, but not to the control substance, despite having used the same test system containers in both conditions.

While it is true that there are additional components in many lavender preparations, we have not found prepubertal gynecomastia to be associated with other non-lavender-containing colognes or topical products. Thus, lavender itself is a logical and well-founded explanation for the physical findings in our patients.

Regarding tea tree oil, I previously evaluated a patient who was exposed over a period of several years to numerous products containing tea tree oil, including shampoos, toothpaste, detergents, home cleaning products, and melaleuca essential oil for minor cuts, burns, and other skin issues. He developed severe prepubertal gynecomastia that improved upon discontinuation of the exposure to these products. As I did in the cases of lavender exposure, I conducted a complete endocrine evaluation and his hormone levels were all within normal ranges. I did not publish this case at the request of his parents, who were in the tea tree oil industry. As clinicians, if we find patients with prepubertal gynecomastia who have been exposed for prolonged periods of time to substances known to activate the estrogen receptor, it provides substantial evidence of causality.

The possibility exists that there are contaminants that have estrogenic activity in these preparations. However, it is the responsibility of the industry that the products as sold are safe for human exposure.

It is, however, of utmost importance to publish these case reports to offer the balance you refer to, and to deepen the scientific knowledge base and protect consumers from unnecessary harm. Clinicians must be informed and use their clinical judgment to determine what is best for their individual patients.

Alejandro Diaz, M.D. and Marco Danon, M.D. Pediatric Endocrinologists

On 2015 Sep 14, Tony Larkman commented:

It is disappointing that the authors didn’t include or at least refer to the following two articles in this paper: 1. Politano VT et al (2013), Uterotrophic Assay of Percutaneous Lavender Oil in Immature Female Rats; International Journal of Toxicology; 32(2): 123-9 (http://www.ncbi.nlm.nih.gov/pubmed/23358464) 2. Carson CF et al. (2014), Lack of evidence that essential oils affect puberty; Reproductive Toxicology; 44:50-1 (http://www.ncbi.nlm.nih.gov/pubmed/24556344)

The first paper largely exonerates lavender oil through the ’gold standard’ uterotrophic assay while the second hypothesizes a mechanism that may be causal in incidences of both gynecomastia and premature thelarche as well as the ‘in vitro’ findings of Henley et al (2007) where they in fact used polystyrene in their test system and not glass. Based on these it is not impossible to conceive that other endocrine disruptors may have been inadvertently present in the material tested.

A further, as yet untested, hypothesis for these manifestations is the high incidence of adulteration in essential oils. The incidence of adulteration in TTO is remarkably high as reported in Wong YF et al. (2014) Enantiomeric distribution of selected terpenes for authenticity assessment of Australian Melaleuca alternifolia oil; Industrial Crops & Products; 67: 475-83 (http://ijt.sagepub.com/content/early/2013/01/24/1091581812472209) where more than 50% of 43 commercial samples tested failed to comply with the proposed chiral ratios. Of 15 commercially sourced samples in the European Union 73% of these showed significant differences in chiral abundances while TTO from both North America and Asia also displayed similar results where ≥50% of the tested samples did not match the expected results. In material of Chinese origin the incidence rises to 100%. An extraordinary range of compounds never found in pure TTO has been coincidentally detected so it has been further hypothesized that this, along with the likelihood that the material used is heavily oxidized, may be significant source of problems such as allergic contact dermatitis attributed to the use of TTO as well as conditions related to endocrine disruption.

It is disappointing for the tea tree oil industry as a whole that TTO continues to be mentioned as a potential endocrine disruptor without a more balanced view being presented when there are clear indications available in the literature that the original proposal by Henley et al in 2007 may be flawed. The fact that TTO was not present at all in any of the material tested is also disappointing as TTO can in no way be implicated yet its mention continues to promulgate the link and implicate TTO unfairly.

On 2018 Jan 07, Mark Milton commented:

In defense of the authors, their article was first published online on September 8, 2015 and appeared in the Issue published on April 1, 2016. The BIAL incident occurred in Jan 2016, i.e. after this article was published and hence there couldn't have been any discussion of the BIAL incident in this article.

On 2016 Jun 08, Christopher Southan commented:

Some discussion of the BIA-10-2474 fatality and AEs would have been pertinant, even though it was more recent than the cut-off date https://cdsouthan.blogspot.se/2016/01/the-unfortunate-case-of-bia-10-2474.html

On 2015 Dec 07, Sanjay Kumar commented:

We appreciate the interest in our work. While we are unable to comment on the internalization mechanisms of the SmartFlare reagents based on our data, our independent RT-PCR measurements (Fig. 3) are consistent with the SmartFlare data and confirm that matrix stiffness and ligand density regulate miR18a levels in this system.

On 2015 Nov 18, Raphael Levy commented:

This article uses the SmartFlare technology to detect miR18a. I would be most interested in the authors' opinion on how these probes escape endosomes to report on miR18a level. In our experience, they do not and fluorescence increase is most likely correlated with degradation in endosomes. It might be quite plausible that endocytosis "is non-linearly regulated by matrix stiffness and fibronectin density in glioma cells."

On 2015 Nov 12, Christopher Sampson commented:

It seems that personalised screening for breast cancer is a sheep in wolves' clothing (rather than the author's suggestion of the reverse).

The purpose of risk-based screening is to redirect resources to those at the greatest risk of disease; those in the greatest need. I have argued elsewhere that it is possible to prioritise screening in this way, in order to maximise the benefits of screening within a given budget.

Feig's commentary suggests that the terms "risk-based screening" and "personalized screening" are being misappropriated. That may well be true. However, the criticisms therein relate to very specific uses of these terms and to particular guidelines (I do not see these claims holding true more broadly). It does not then follow that 'risk-based screening' and 'personalisation' are wolves, and indeed the way they have been clothed does not appear desirable. They are sheep in wolves' clothing!

We should not -- as I hope Feig would agree -- allow the unsavoury dressing-up of these ambitious developments waylay further research into risk-based screening.

On 2017 Feb 18, Clive Bates commented:

Erroneous interpretations have been placed on these results and overly confident conclusions drawn from very small numbers of imperfectly characterised teenagers. The headline recommendations were based on the behaviour of six out of 16 baseline e-cigarette users in a sample of 694 adolescents deemed not to be susceptible to smoking. Large conclusions drawn from small numbers should always be a cause for caution, as discussed in this article about this study by Five Thirty-Eight:

Ignore The Headlines: We Don’t Know If E-Cigs Lead Kids To Real Cigs by Christie Ashwandan, 11 September 2015

One should expect the inclination to use e-cigarettes to be caused by the same things that cause an inclination to smoke - they are similar habits (the former much less risky) and it is quite likely that those who used e-cigarettes first would have become smokers first in the absence of e-cigarettes - a concept known as shared liability. A range of independent factors that create a common propensity to smoke or vape, such as parental smoking, rebellious nature, delinquency etc. explain the association between vaping and smoking incidence but without this relationship being causal.

The authors try to address this by characterising teenagers non-susceptible to smoking if they answer “definitely no” when asked the following: “If one of your friends offered you a cigarette, would you try it?” and “Do you think you will smoke a cigarette sometime in the next year?”. The study concentrates on this group.

This is not a foolproof way of characterising susceptibility to smoking, which in any case is not a binary construct but a probability distribution. Nor is susceptibility a permanent condition for any young person - for example, if a teenage girl starts seeing a new boyfriend who smokes that will materially changes her susceptibility to smoking. The fact that some were deemed unsusceptible to smoking but were already e-cigarette users is grounds for further unease - these would be more likely to be the teens where the crude characterisation failed.

It is a near-universal feature of tobacco control research that the study presented is a wholly inadequate basis for any policy recommendation drawn in the conclusion, and this study is no exception:

These findings support regulations to limit sales and decrease the appeal of e-cigarettes to adolescents and young adults.

The findings do not support this recommendation, not least because the paper is concerned exclusively with the behaviour of young people deemed not susceptible to smoking and, within that group, a tiny fraction who progressed from vaping to smoking. Even for this group (6 of 16) the authors cannot be sure this isn't a result of mischaracterisation and that they would not have smoked in the absence of e-cigarettes. The approach to characterising non-susceptibility is far too crude and the numbers involved far too small to draw any policy-relevant conclusions.

But this isn't the main limitation. Much more troubling is that the authors made this policy recommendation without considering the transitions among young people who are susceptible to smoking - i.e. those more likely to smoke, and also those much more likely to use e-cigarettes as well as or instead of smoking. This group is much more likely to benefit from using e-cigarettes as an alternative to smoking initiation, to quit smoking or cut down or as a later transition as they approach adult life.

There are already findings (Friedman AS, 2015, Pesko MF, 2016, Pesko MF, Currie JM, 2016) that regulation of the type proposed by the authors designed to reduce access to e-cigarettes by young people has had unintended consequences in the form of increased smoking - something that should not be a surprise given these products are substitutes. While one may debate these findings, the current study makes no assessment of such effects and does not even cover the population that would be harmed by them. With these limitations, it cannot underpin its own over-confident and sweeping policy recommendation.

On 2015 Sep 15, Aadil Inamdar commented:

The very aim of a systematic review is to provide a critical summary of the available evidence. The authors of a systematic review may draw conclusions based on the collected studies on a particular topic. It is equally important to present to their readers an unbiased quality assessment of the included studies in the review, which is an integral part of a systematic review process. This process is what differentiates a systematic review from the rest and a missing quality assessment of included studies section brings it as par with the common narrative or a literature review. The risk of bias and quality assessment of studies included in this review is missing. The onus is on the researchers to provide clear, simple conclusions of studies that has been meticulously checked and rechecked for their validity. Otherwise, on the contrary, it only helps to pollute the current evidence.

It is important to note that;

• Systematic reviews are one of the highest levels of information in the hierarchy of evidence LINK
• Steps involved in a systematic review have been laid down by the Cochrane collaboration STEPS
• A major section involves assessment of quality and risk of bias, of the included studies
• Assessment not only helps in ascertaining validity of a particular study, but also gives a true estimate (over/under) of the intervention (or exposure) Assessment of quality and risk of bias

On 2015 Nov 14, Arnaud Chiolero MD PhD commented:

An enlightening review revealing the issues of universal lipid screening in children. Similar issues with the screening of other CVD risk factors in children (see www.publichealthreviews.net/content/36/1/9)

On 2015 Sep 10, David Keller commented:

Did droxidopa increase dyskinesias, blood pressure, heart rate & adrenergic side-effects?

In general, medications which improve rigidity and bradykinesia and other "off symptoms" in Parkinson disease (PD) patients also tend to worsen dyskinesias. If droxidopa was able to improve PD "off" symptoms without worsening dyskinesias, then this drug is a breakthrough. The effect of droxidopa on dyskinesias is a crucial outcome and should be reported in the abstract. Without this information, the study cannot be evaluated properly.

Droxidopa is a metabolic precursor of norepinephrine, so it is expected to result in adrenergic effects, such as increasing blood pressure, heart rate, insomnia, heart arrhythmias, constipation, etc. The presence or absence of such side-effects is of interest to clinicians and patients, and should also be reported in the abstract.

Addendum

On September 16, 2015, I received a very informative email from Shifu Zhao, MD, in which he stated that "adrenergic side-effects were not significantly increased comparing with placebo group or baseline data at droxidopa dose 600mg/day" including "blood pressure, insomnia, palpitation, ECG" [1].

In addition, he assured me that add-on droxidopa therapy improved tremor and alternating motion of the hands in patients with moderate-to-severe Parkinson's disease, without worsening dyskinesias. [1]

I thank Dr. Zhao for supplying this information to me, and I hereby relay it to other Parkinson disease patients who read scientific abstracts without access to the underlying journal articles.

Reference:

1: Zhao S, Personal Communication by email, Received 9/16/2015 in reply to an earlier email inquiry.

On 2017 Mar 13, Atanas G. Atanasov commented:

Excellent overview on the topic, I have featured this publication at: http://healthandscienceportal.blogspot.com/2017/03/the-stomach-in-health-and-disease.html

On 2015 Oct 26, Gustav van Niekerk commented:

It would be hard to over emphasise the intense selective pressure that pathogens place on host populations (as predicted by the Red Queen hypothesis). Our genome are littered with pseudonized genes formerly associated with immune-related functions. These ‘genomic fossils’ representing discarded immunological innovations that became redundant as our pathogens evolve strategies to defeat them, testifying to an ancient conflict that have been ranging on between host and pathogens. Sequencing genomes have repeatedly demonstrated that genes with immunological functions tend to be under selective pressure and tend to be highly polymorphic (Mother Nature is ‘diversifying’ the ‘portfolio’ of immunological strategies: a pathogens can overcome many immunological strategies, but not all of them simultaneously. This observed immunological heterogeneity in population is how a species ‘hedge it’s bet’ against pathogens). Host and pathogen/parasites occasionally demonstrate "mirror-image phylogenies", demonstrating the close coevolution as pathogens keep in step with host development. Collectively, such observations suggest that pathogens exerts immense evolutionary pressure and one of the biggest drivers for evolutionary novelty.

Consequently, we suggest a less exiting narrative in which an AIS evolved initially in response to pathogen stress. Similar to your work Corcos D, 2015, we also noted that AIS evolved in an aquatic environment. Sediment and marine environments typically have a higher pathogen burden (see references in < PMID: 25698354 >), thus suggesting a higher level of pathogen stress. However, as you point out, there are scepticism about whether the AIS represent a true immunological innovation, as invertebrates do not seem any worse-off than vertebrates in suffering from infections. In this regard, we would like to point out that the AIS could have been an initial innovation that was subsequently overcome by fast evolving pathogens. That is, the AIS could have evolved in response to pathogen stress, providing an initial benefit to vertebrates, but was subsequently overcome by pathogens. And now, regardless whether the AIS is currently an ‘immunological innovation’, we are stuck with it (for better or worse). Finally, it is not obvious that the AIS do not repeat a lasting innovation. After all, “[v]ertebrates are the dominant group of animals on the Earth, given their abundance, large body sizes and presence at the top of both aquatic and terrestrial foodwebs” Wiens JJ, 2015. How did vertebrates manage to survive (and indeed flourish) in a world initially dominated by invertebrates? An AIS may have some additional benefits that have helped vertebrates invade into niches previously occupied by invertebrates. How AIS could provide a benefit that is (a) only applicable to vertebrates and (b) does not involve an enhancement of immune potency is currently an open question.

(PS we unfortunately do not have institutional access to the article from Burnet FM. We will shortly be sending a letter to the editor, commenting on your Corcos D, 2015 interesting article)

On 2015 Oct 20, Daniel Corcos commented:

This paper is interesting for examining the relationship between the adaptive immune system and the vascular system. However it does not explain such a complexity with no obvious advantage. For instance, immunodeficiency in zebrafish does not result in a dramatic change in viability. This can be interpreted in the words of Hedrick as an indication that the AIS is "not so superior"(1) or in the words of Burnet that it is "related to something other than defence against pathogenic microorganisms."(2) I have proposed (3) that its origin was related to intraspecific predation (cannibalism).

1)Hedrick SM. Immune system: not so superior. Science 2009;325:1623–4.

2)Burnet FM. “Self-recognition” in colonial marine forms and flowering plants in relation to the evolution of immunity. Nature 971;232:230–5.

3)Corcos D. Food-Nonfood Discrimination in Ancestral Vertebrates: Gamete Cannibalism and the Origin of the Adaptive Immune System. Scand J Immunol. 2015;82(5):409-17.

On 2017 Jul 04, Randi Pechacek commented:

Alex Alexiev wrote a blog post on microBEnet, mentioning this paper, arguing the importance of equipment cleanliness for food quality.

On 2016 Jan 20, thomas samaras commented:

A very interesting study. However, the conclusion that overweight may not be harmful at older ages is not consistent with what we know about the dangers of increasing BMI and various health parameters. Lamon-Fava showed many years ago that virtually all biological parameters get worse with increasing BMI in a linear trend; e.g., BP, fibrinogen, Apo B, Apo A, and HDL. In addition, the New England Centenarian Study found that almost all men who reached 100 years of age were lean. The Okinawan centenarians are also very lean. We also know that pre-western people with low chronic disease throughout their lives also tend to be quite lean in old age; e.g., people of Kitava (near Papua New Guinea).

Maier, Van Heemst and Westendorp found that shorter 85 and 90 years were more likely to reach advanced ages. These shorter, longer living individuals also had longer telomeres than taller people as found in the Guzzardi, et al. study. Perhaps confounding occurred by a number of lean people who were developing health problems without current symptoms, smokers, or poorly nourished individuals. This confounding may explain the results in regard to the positive correlation of increased % of fat mass and cholesterol between the start and end of the study.

On 2018 Jan 19, Natalie Clairoux commented:

Free version of this article available at http://hdl.handle.net/1866/19740

On 2016 Feb 10, Anders von Heijne commented:

In a recent case with a very large testicular epidermoid cyst we found an ADC-value of 0.544. IT seems that testicular epidermoid cysts have distinctly lower ADC-values than their intracranial counterparts.

On 2018 Jan 19, Natalie Clairoux commented:

Free version of this article available at http://hdl.handle.net/1866/19736

On 2015 Dec 08, S. Celeste Morley commented:

Thank you very much for your interest in and comment upon our work. The safety and efficacy of PCV in preventing and reducing the incidence of invasive pneumococcal disease is unquestioned and unequivocally supported by all literature. The mechanisms by which PCV protects are multifactorial. PCV generates an immune response that protects against invasive disease, if not colonization. We did not address incidence of IPD in this study. PCV also results in decreased carriage prevalence of the disease-causing serotypes covered by the vaccine, and thus shifts serotype prevalence without necessarily altering overall carriage prevalence. Our study simply reported overall carriage prevalence and the antibiotic susceptibility profiles of carriage isolates; we did not report serotypes. Our results finding that overall likelihood of colonization with any pneumococcal serotype is not affected by PCV is in line with other larger studies (e.g. Zuccotti et al Vaccine. 2014 Jan 23;32(5):527-34. Zuccotti G, 2014) Thus, our study is not in conflict with any of the studies referenced above, which clearly show a decrease in colonization with vaccine-covered serotypes.

On 2015 Dec 07, Manoochehr Karami commented:

Manoochehr Karami, PhD, Research Center for Health Sciences and Department of Epidemiology, School of Public Health, Hamadan University of Medical Sciences, Hamadan, Iran.

In an interesting study published recently, Julie Y. Zhou et al.(1) highlighted the prevalence of nasopharyngeal pneumococcal colonization among children in the greater St. Louis hospital. Authors have stated "pneumococcal conjugate vaccine (PCV) did not alter prevalence" of nasopharyngeal carriage. World Health Organization indicated that after PCV introduction, both targeted and non-targeted vaccination population were affected by direct and indirect effects of PCV immunization(2). Moreover, published studies (3-7) supported the changes of epidemiological profile of Streptococcus pneumonia related diseases transmission and nasopharyngeal carriage even among those individuals who were not immunized against Streptococcus pneumonia. Accordingly, it seems Zhou JY et al. interpretations based on their findings is in question and might be affected because of potential selection bias while enrolled participants. Rational for such selection bias is the catchment area of St. Louis hospital and potential differences between study participants and non-participants. Although they have excluded some patients, however this strategy does not guarantee the representativeness of their own work. Generally speaking, better explanation for Zhou et al findings is selection bias. In conclusion, lack of generalizability of this study findings should be considered by policy makers and interested readers.   References: 1. Zhou JY, Isaacson-Schmid M, Utterson EC, et al. Prevalence of nasopharyngeal pneumococcal colonization in children and antimicrobial susceptibility profiles of carriage isolates. International Journal of Infectious Diseases;39:50-52. 2. World Health Organization. Measuring impact of Streptococcus pneumoniae and Haemophilus influenzae type b conjugate vaccination. WHO Press, Geneva, Switzerland, 2012. 3. World Health Organization.Pneumococcal vaccines WHO position paper – 2012.Wkly Epidemiol Rec. 2012;87:129-244. . 4. Lehmann D, Willis J. The changing epidemiology of invasive pneumococcal disease in aboriginal and non-aboriginal western Australians from 1997 through 2007 and emergence of nonvaccine serotypes. Clinical Infectious Diseases. 2010, 50(11):1477–1486. 5. Pilishvili T, Lexau C. Sustained reductions in invasive pneumococcal disease in the era of conjugate vaccine. The Journal of Infectious Diseases, 2010, 201(1):32–41. 6. Davis S, Deloria-Knoll M, Kassa H, O’Brien K. Impact of pneumococcal conjugate vaccines on nasopharyngeal carriage and invasive disease among unvaccinated people: Review of evidence on indirect effects. Vaccine.2014;32:133-145. 7. Karami M, Alikhani MY. Serotype Replacement and Nasopharyngeal Carriage Due to the Introduction of New Pneumococcal Conjugate Vaccine to National Routine Immunization. Jundishapur Journal of Microbiology 2015;8.

On 2015 Sep 10, George McNamara commented:

Over at PubPeer (where this will be mirrored to) there were questions as to why I consider the image quality of figures 1 and 2 are so bad. See the Ma et al 2013 paper - also published in PNAS. http://www.pnas.org/content/110/52/21048.figures-only

Was not that hard two years ago to publish high quality images. Nor was it that hard for Deng et al to publish good quality figure 3 or supplemental files.

On 2015 Sep 09, George McNamara commented:

The good news of this paper: open access. The bad news: figures 1 and 2 are among the worst light microscopy image quality I have seen in a long time. I don't know if PNAS completely dropped the ball on image quality (fig 3 does look ok - also the authors do get to approve or fix the page proofs), or something to do with the authors. When I was a graduate student, I thought the difference between a cell biologist and a biochemist was that a cell biologist (or their lab) had microscopes and knew how to use them, and biochemists did not and did not care. Alternative hypothesis: PNAS figures are managed by biochemists. Considering the authors are down the hall from Eric Betzig, who won a prize of high resolution microscopy (and has since published two Science papers on more high res microscopy), they could have acquired and published better images. Their centromere, telomere repeats, MUC1 and MUC4 tandem repeat targets were all previously published (and cited by them) with somewhat better image quality.

On 2015 Oct 01, Badrul Arefin commented:

Dear readers,

We found two mistakes either as a missing vertical axis title or a typo in the following figures in this published article. In figure 7A, vertical axis title is missing when figure re-arrangement is performed in a revised version. Here it should be "Eclosure (%)". In figure 8I, N-NAME in chart area should be replaced with "L-NAME". Please excuse us for these mistakes. However, both of these are correctly-written/correct in the respective figure legends and elsewhere in the article.

Sincerely,

Badrul Arefin, On behalf of all authors.

On 2015 Sep 01, Friedrich Thinnes commented:

From my point of view the dust-raising effect of antibody 6E10 fuels the idea that plasmalemma-integrated VDAC-1 works as a receptor of amyloid Aß.

It has been shown that docking of amyloid Aß via GxxxG motif interaction to cell membrane-standing VDAC-1 opens the channel, and this way induces neuronal cell deaths accumulating over time.

Whenever critical brain regions and all redundant structures are affected Alzheimer´s Dementia appears.

For details see Friedrich P. Thinnes, Biochimica et Biophysica Acta 1848 (2015) 1410–1416 and www.futhin.de

On 2015 Sep 11, Chanin Nantasenamat commented:

Thanks Christopher for the reminder. Actually, we also wanted to make the dataset publicly available. Thus, please find the dataset used in this study (in Excel file format) from the following link:

http://dx.doi.org/10.6084/m9.figshare.1539580

On 2015 Sep 05, Christopher Southan commented:

This is a usefuly detailed study but direct availability of the 2,937 structures should have been a condition of publication. I note they were selected from BindingDB but also filtered post-download (and will change via updates). The authors should thus please surface at least the SMILES on figshare or other open acess option (Sep 14th - good 2 c the file :)

On 2015 Oct 23, Miguel Lopez-Lazaro commented:

This article provides a model to explain that virtually every cancer cell within a tumor often contains the same core set of genetic alterations, with heterogeneity confined to mutations that emerge late during tumor growth.

In my opinion, there is a simpler explanation. Recent evidence strongly suggests that cancer arises from normal stem cells. If cancer arises from normal stem cells, all the mutations occurring in these cells before becoming malignant (cancer stem cells, CSCs) will be found in all their progeny, that is, in all the tumor cancer cells. Some tumor cells may lack some of these mutations if they lose during cell division the chromosomes or pieces of chromosomes that bear these DNA alterations. The mutations arising during the self-renewal of CSCs will be found only in the tumor populations derived from these malignant stem cells. In addition to self-renewing, CSCs generate progenitor cancer cells, which divide and produce the bulk of cancer cells within a tumor. The mutations found in few tumor cancer cells probably occur during the division of these progenitor cells. In some cases, the tumor cancer cells may arise from more than one normal stem cell. In these cases, not all the cancer cells within a tumor will share the same core set of genetic alterations (1). Normal and malignant stem cells are defined by their self-renewal capacity and differentiation potential, and have a natural ability to migrate.

(1). Lopez-Lazaro M. Selective amino acid restriction therapy (SAART): a non-pharmacological strategy against all types of cancer cells. DOI: 10.18632/oncoscience.258

On 2015 Aug 28, Tom Kindlon commented:

Minimal change on only objective outcome measure:

On the only objective measure, the activity monitor, at 52 weeks (compared to baseline): the MRT group had increased by 5.8% and CBT group had increased by 6.6%.

There was no control group. One would expect a no therapy CFS group would likely on average increase their activity a little also in 52 weeks esp. in the early years of the illness and/or in the first year or two after diagnosis.

On 2015 Oct 03, Prashant Sharma, MD, DM commented:

The affiliation for Drs. Varma N, Sachdeva and Sharma should read as "Department of Hematology" and not "Hepatopathology".

On 2015 Sep 08, Alexandra Alexiev commented:

This article was written about in a blog post at: http://microbe.net/2015/09/08/antimicrobial-resistance-countermeasures-in-veal-farming/

MicroBEnet is a microbiology of the built environment blog funded by the Sloan foundation to communicate about scientific research in the field.

On 2015 Nov 26, Lydia Maniatis commented:

We can get a sense of the intellectual poverty of this study simply by reading the "Conclusions" section:

"Our experiments show that luminance edges play a central role in White's illusion. The illusion seems to be predominantly caused by the luminance edge between the test patch and its background bar, while the edge contrast to neighboring bars is largely ignored. The effect of contour adaptation on White's illusion could not be replicated by spatial filtering models, which adds further evidence against the adequacy of such models as a mechanistic explanation of White's illusion in particular, and lightness perception in general. Our results highlight the importance of further investigating the question of how surface lightness is computed from edge contrast. "

There is no content here because:

1) There is no alternative to the idea that "luminance edges play a central role in White's illusion." Both sides of the display are the same, other than the (perceived) location of the targets as lying on white or on black. Note that there is NO REFERENCE to the authors' orientation claims.

2) The inadequacy of a priori inadequate models that weren't even properly tested (assuming they could be) is not an argument for anything.

3). The statement that "Our results highlight the importance of further investigating the question of how surface lightness is computed from edge contrast" is meaningless. If there are questions to be answered, this study did not address them nor make them seem any more interesting.

The ad hotness of the authors' orientation claims is not only refuted by other displays, such as the Benary cross, but can also be refuted by versions of White's illusion in which the edges of the targets are curved so as to collectively be consistent with, e.g. an interrupted circle. Try it. Such a display, for me, makes more evident the fundamentally bistable character of the lighter-looking group of targets. They can either appear to be part of an opaque surface that lies behind the white stripes, on an amodally-completed black background, or they can appear to form part of a transparency that passes over the black and white stripes. The transparency aspect of White's illusion, which has been noted before, and is noticed by naive observers, is, of course, never touched on in this paper.

On 2015 Nov 08, Lydia Maniatis commented:

This article should probably not be read from the beginning, but from the end – from the last section of the discussion, titled “Difficulties with matching tasks.” The major theoretical problems discussed earlier seem moot once you appreciate that problems with method render the data virtually worthless..

The stimulus effects were not robust but, rather, highly ambiguous and liable to produce theoretically relevant effects that the authors did not bother to consider. This led to highly variable responses that they could not interpret. The authors try and put the blame for their data problems on the lightness matching technique itself, but the fault lies in them: If a task (or a stimulus) is not fit for purpose, it is the fault of those who chose it when it fails to deliver.

The problem is, in fact, the stimuli, which, again, were highly unstable and often did not produce the percepts that the authors predicted and which needed to arise reliably in order to allow them to validate their predictions. Four of ten observers, we are told, did not even perceive White's illusion! These observers were “following some strategy, but [our data] does not allow us to understand what exactly they are doing.”

In general, there was “large variability across and within observers....With simple stimuli of the kind used here, the perceived lightness of different image parts can change over time....In that case, the mean across trials may not be a good indicator of subjective experience [i.e. perception].” So the authors do not really know what their subjects are perceiving.

It seems, further, that the “simple stimuli” produced perceptual phenomena that the authors were or should have been aware are possible – either through the literature or simply on the basis of looking at their stimuli (the least a perception scientist should be doing when planning an experiment is to notice obvious effects of their stimuli) - but did not take into consideration: “Ekroll and Faul (2013) observed an additional transparent layer for similar stimuli...” This might “explain the inconsistent results...” So the “simple stimuli” could produce complex percepts - precepts that, for these authors simply amounts to noise.

In addition: “...observers sometimes selected match lightnesses that were outside the luminance range spanned by the grating. This is surprising....” They really have no idea what their data mean.

The authors' stunning conclusion: “This is just one further example that lightness matching is not such an easy and straightforward task as it might appear on the surface.” They don't seem to realize that no method is secure if your stimuli are ambiguous and unstable and you have not inspected them and factored the potential effects into your theory and method.

Unfortunately, lesson seems not to have been learned, as revealed by the final discussion section of Betz, Shapley, Wichmann and Maertens, 2015, which similarly describes devastating methodological problems as an afterthought. Why are reviewers setting such an obviously low bar for publication?

On 2015 Aug 30, Lydia Maniatis commented:

The authors of this study construct an untenable, ad hoc account of a myopic observation.

The observation is that the contrast effect in White's illusion “is largely determined by edge contrast across the edge orthogonal to the grating, whereas the parallel edge has little or no influence.”

This is a correct literal description of White's illusion. If we had no knowledge of any other contrast illusions, we might overgeneralize from this and conclude that, in general, orthogonal edges produce contrast effects while parallel edges do not. But we do know more. We know that contrast effects are not tied to edge orientation in this way. Consequently we would not reasonably attempt to construct a neural model of contrast based on a principle of “Orthogonal edges produce contrast, parallel edges don't,” since such an ad hoc model, shrink-wrapped to fit White's illusion, would instantly be challenged and falsified by any number of other contrast effects. However, this is precisely what Betz et al propose. (Not to mention that, being “low-level,” the authors' proposed mechanism would also have to be compatible with every other “low-level” as well as every “high-level” effect, e.g. transparency effects, since information from the early parts of the visual system is the basis for the high-level effects.)

In addition, the authors tailor their account to a very weak version of White's illusion (as the authors show, its effect is comparable to the classic slc demo), composed of single target bars rather than columns of aligned bars. The latter produce a much larger lightness differences as well as transparency effects (in the case of the lighter-seeming bars). Does the proposed model work for this classic version as well? Does it work for the classic simultaneous contrast illusion? Does it work for round targets on square backgrounds? Does it explain the Benary cross? If not, then how does such an account contribute to theoretical or practical understanding of the phenomenon (lightness contrast) under consideration? What are the chances that the visual system possesses a mechanism especially for producing a version of White's illusion?

As far as their experimental observations, the authors take a little too much credit for themselves when they say that their experiments have shown that “not all luminance borders that enclose a surface are treated equally in White's illusion.” This is an unnecessarily narrow framing of a many-decades-known, fundamental fact of lightness perception.

If there is one safe statement we can make about simultaneous contrast, it is that it is closely correlated with the appearance of a figure-ground relationship. Specifically, the surface that appears as figure lightens against a darker (apparent) background, and darkens against a lighter one. Mere (apparent) adjacency does not produce contrast. So when the authors claim to have shown that “the luminance step between the test patch and the grating bar on which it is [apparently!] placed is the critical condition for perceiving White's illusion” they a. Have not shown anything new and b. seem naive to the theoretical implications and generality of their own casual description (“on which it is placed”).

The implications are that, given the robustness of the figure-ground/simultaneous contrast link, the challenge of predicting simultaneous contrast reduces to the challenge of predicting figure-ground relations. The latter, in contrast, is not reducible to structure-blind theories involving inhibition/filtering based on luminance ratios. And it is certainly not reducible to an orthogonal/parallel edge dichotomy.

On 2015 Oct 07, Bart Verkuil commented:

An Associate Editor of The Journal of The Norwegian Medical Association made us aware of an error in the Introduction of this paper: we mention that a meta-analysis of the longitudinal relation between workplace bullying and mental health, published in The Journal of The Norwegian Medical Association (Tidsskrift for Den norske legeforening) in 2014, is only available in the Norwegian language. This is incorrect, as it is available in English: http://www.tidsskriftet.no/article/3213422

On 2016 Aug 23, Ben Goldacre commented:

This trial has the wrong trial registry ID associated with it on PubMed: both in the XML on PubMed, and in the originating journal article. The ID given is NCT013655104. We believe the correct ID, which we have found by hand searching, is NCT01365104.

This comment is being posted as part of the OpenTrials.net project^[1] , an open database threading together all publicly accessible documents and data on each trial, globally. In the course of creating the database, and matching documents and data sources about trials from different locations, we have identified various anomalies in datasets such as PubMed, and in published papers. Alongside documenting the prevalence of problems, we are also attempting to correct these errors and anomalies wherever possible, by feeding back to the originators. We have corrected this data in the OpenTrials.net database; we hope that this trial’s text and metadata can also be corrected at source, in PubMed and in the accompanying paper.

Many thanks,

Jessica Fleminger, Ben Goldacre*

[1] Goldacre, B., Gray, J., 2016. OpenTrials: towards a collaborative open database of all available information on all clinical trials. Trials 17. doi:10.1186/s13063-016-1290-8 PMID: 27056367

* Dr Ben Goldacre BA MA MSc MBBS MRCPsych<br> Senior Clinical Research Fellow<br> ben.goldacre@phc.ox.ac.uk<br> www.ebmDataLab.net<br> Centre for Evidence Based Medicine<br> Department of Primary Care Health Sciences<br> University of Oxford<br> Radcliffe Observatory Quarter<br> Woodstock Road<br> Oxford OX2 6GG

On 2016 Apr 22, Falk Leichsenring commented:

For a response to Hofmann, see http://ebmh.bmj.com/cgi/content/full/eb-2016-102372?ijkey=giCBgiE7JTlTg&keytype=ref&siteid=bmjjournals

On 2016 Mar 25, Stefan Hofmann commented:

For a more detailed critique and response, see: http://www.ncbi.nlm.nih.gov/pubmed/27009055

On 2016 Feb 27, Falk Leichsenring commented:

We absolutely agree: strong conclusions require strong evidence

Falk Leichsenring, Patrick Luyten, Mark J. Hilsenroth, Allan Abbass, Jacques P. Barber, John R. Keefe, Frank Leweke, Sven Rabung, Christiane Steinert

Hofmann [1] asked us to provide evidence - here it is.

In our first response [2] to Hofmann et al. [3], we showed that their commentary on the quality of studies of PDT is not consistent with empirical evidence from quality research performed in adversarial collaboration between PDT and CBT researchers, which found no significant differences in quality between studies of PDT and CBT [4, 5]. Furthermore, we noted that Gerber et al. did not find significant correlations between methodological quality and outcome in studies of PDT [5]. These results (i.e., evidence) are inconsistent with Hofmann et al.’s conclusions [3], regardless of whether Hofmann et al.’s ratings [3] are methodologically sound.

In addition, we emphasized that Hofmann et al. failed to demonstrate any evidence that the quality of the 64 RCTs leads to results in favor of PDT. In a similar way Bhar and Beck suggested that the lack of difference in outcome between CBT and PDT found by Leichsenring, Rabung, and Leibing [6] was due to poor treatment integrity [7]. However, using Bhar and Beck’s own integrity ratings, their assertion was not corroborated by empirical data [8]. It is of note that these meta-analyses [6, 8] included researchers from both CBT (E.L.) and PDT (e.g. F.L.).

In our first response, we also observed that the authors failed to provide basic data on interrater reliability, raters’ training, the rating procedures, attempts to address allegiance effects, or blinding of raters [2]. The authors also did not include researchers of both approaches among the raters, as done by Gerber et al. and Thoma et al. [4, 5]. In addition, we noted that Hofmann et al. based their conclusions of poor methodological quality on "unclear" designations of quality [2]. Most authors would have attempted to contact the original authors of a study before asserting that procedural information was unclear and making strong conclusions about study quality.

Hofmann et al. [3] drew strong conclusions about the quality of our review using extreme terms such as "invalidating the authors´ results" and "making the findings meaningless" using nonstandard procedures of questionable quality. For strong conclusions, strong evidence is required. Yet, Hofmann et al. failed to provide it. For a commentary aiming to address study quality, it is puzzling to apply procedures of such poor quality.

We are raising these issues again since Dr. Hofmann did not address them in his response [1]. Instead of doing so, Dr. Hofmann stated [1]: ”As their only defense, the authors argue that CBT is also poorly supported." In this way, he is simply ignoring the evidence we provided and the methodological shortcomings of his commentary we had pointed out [2]. Further, we did not question that CBT is an efficacious treatment. We just pointed out that the available evidence shows that the quality of CBT studies is no better than that of PDT studies [4, 5].

We also did not intend to attack Dr. Hofmann on a personal level, but rather intended to provide evidence that he repeatedly applied double standards when judging studies of CBT as compared to those of PDT [2, 9, p. 49-51]. We respectfully asked that if he chooses to write about PDT (e.g., comment on a meta-analysis, conduct a meta-analysis, or conduct a study involving PDT), that he considers involving a psychodynamic researcher in the process. This invitation still stands.

Dr. Hofmann emphasized that CBT is widely disseminated in the UK. This is true, but PDT is recommended by treatment guidelines and implemented in the National Health Service in the UK as well. This is also true in other countries. In Germany, for instance, PDT is as frequently used as CBT [10]. The Scientific Board for Psychotherapy (Wissenschaftlicher Beirat Psychotherapie; WBP) is the paramount body in Germany for assessing the scientific status of psychotherapeutic interventions. For this purpose, standardized and transparent criteria are used. Based on a careful evaluation by the WBP, both CBT and PDT were acknowledged as scientific and efficacious forms of psychotherapy (www.wbpsychotherapie.de). It is noteworthy that the WBP is composed of researchers from diverse psychotherapeutic orientations (e.g. CBT, PDT, and systemic therapy). The studies of PDT were evaluated by CBT researchers, and vice versa. The conclusions by a balanced expert institution such as the WBP are incompatible with those by Hofmann et al. [3].

We all should be happy that a variety of psychotherapeutic treatments exist that are beneficial to patients. Future research should address the question of which patients benefit most from which treatments, and why.

Declaring the evidence of a whole treatment approach as "meaningless" is not supported by the preponderance of evidence, and is counter-productive to this goal.

References

1 Hofmann SG: Show us the data! Pubmed Commons Feb 17 2016 2 Leichsenring F, Luyten P, Hilsenroth MJ, Abbass A, Barber JP, Keefe JR, Leweke F, Rabung S, Steinert C: Once again: Double standards in psychotherapy research - response to hofmann et al. PubMed Commons 2016 3 Hofmann SG, Eser N, Andreoli G: Comment from pubmed commons. . January 23rd, 12:28am UTC 2016 2016 4 Thoma NC, McKay D, Gerber AJ, Milrod BL, Edwards AR, Kocsis JH: A quality-based review of randomized controlled trials of cognitive-behavioral therapy for depression: An assessment and metaregression. Am J Psychiatry 2012;169 5 Gerber AJ, Kocsis JH, Milrod BL, Roose SP, Barber JP, Thase ME, Perkins P, Leon AC: A quality-based review of randomized controlled trials of psychodynamic psychotherapy. Am J Psychiatry 2011;168:19-28. 6 Leichsenring F, Rabung S, Leibing E: The efficacy of short-term psychodynamic psychotherapy in specific psychiatric disorders: A meta-analysis. Arch Gen Psychiatry 2004;61:1208-1216. 7 Bhar SS, Beck AT: Treatment integrity of studies that compare short-term psychodynamic psychotherapy with cognitive-behavior therapy Clin Psychol-Sci Pr 2009;16:370-378. 8 Leichsenring F, Salzer S, Hilsenroth M, Leibing E, Leweke F, Rabung S: Treatment integrity: An unresolved issue in psychotherapy research. Curr Psych Rev 2011;7 313-321. 9 Leichsenring F, Rabung S: Double standards in psychotherapy research. Psychother Psychosom 2011;80:48-51. 10 Albani C, Blaser G, Geyer M, Schmutzer G, Brähler E: Outpatient psychotherapy in germany from the patient perspective: Part 1: Health care situation Psychotherapeut 2010;55:503-514.

On 2016 Feb 17, Stefan Hofmann commented:

Show us the data! The authors wrote in the abstract that psychodynamic therapy is as efficacious as treatments established in efficacy. Strong conclusions require strong evidence. Dr. Leichsenring and colleagues were unable to provide the reader with such evidence. Our earlier commentary of the article by Leichsenring and colleagues reported data suggesting that the majority of studies included in their review were of low quality. It is the responsibility of Leichsenring and colleagues to provide the reader with evidence that their conclusions are justified. Instead of providing the reader with such evidence, the authors chose to attack me on a personal level. Whether or not Dr. Leichsenring and colleagues believe that I am adequate to serve as a reviewer or editor of scientific journals or grants and collaborate with others is completely unrelated to the weaknesses of their study. As their only defense, the authors argue that CBT is also poorly supported. The authors are incorrect. The supporting evidence of CBT is overwhelmingly large. Our own review identified 269 meta-analytic studies of CBT. We observed that the quality of studies that entered some of these meta-analyses were not uniformly high. However, some of them were of high quality (e.g., Hofmann & Smits, 2008). Because CBT has such a solid empirical basis, many countries, including the UK, disseminate CBT on a large-scale basis, e.g., http://www.iapt.nhs.uk/. It should be noted that this dissemination is not limited to CBT but also includes other empirically supported treatments. Polemics and personal attacks on my scientific integrity are not the real problem. The biggest concern in my view is that these disputes distract from the real issue. They confuse our patients and policy makers, inhibit scientific progress, and inflict harm by withholding effective treatments. References 1. Hofmann, S. G., Asnaani, A., Vonk, J. J., Sawyer, A. T., & Fang, A. (2012). The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive Therapy and Research, 36, 427-440. doi 10.1007/s10608-012-9476-1 2. Hofmann, S. G. & Smits, J. A. J. (2008). Cognitive-behavioral therapy for adult anxiety disorders: A meta-analysis of randomized placebo-controlled trials. Journal of Clinical Psychiatry, 69, 621-632. doi: 10.4088/JCP.v69n0415

On 2016 Jan 29, Falk Leichsenring commented:

Once again: Double standards in psychotherapy research - response to Hofmann et al.

Falk Leichsenring, Patrick Luyten, Mark J. Hilsenroth, Allan Abbass, Jacques P. Barber, John R. Keefe, Frank Leweke, Sven Rabung, Christiane Steinert

Referring to a recent review on psychodynamic therapy (PDT) [1], Hofmann et al. [2] criticize the quality of studies included in this review. The authors conclude that the "poor quality" of studies of PDT "invalidates" the results of this review making them "meaningless" [2]. The comment by Hofmann et al. [2] deserves some response.

• The conclusions drawn by Hofmann et al. [2] are inconsistent with present research. As shown by an independent research group including proponents of both CBT and PDT working respectfully together, the quality of studies of PDT does not differ significantly from that of studies of CBT which fell into the lower range of adequate quality [3, p. 22, 4]. Most of the studies included by Leichsenring et al. [1] were also included in this comparison [3, 4]. However, Hofmann has not described the respective CBT studies as “meaningless”.

Furthermore, the comment by Hofmann et al. [2] suffers from several shortcomings.

• The authors are incorrect when referring to our publication as a meta-analysis. In fact it was a systematic review [1]. This is of note since possible shortcomings in individual studies would not invalidate the review as a whole.

• The authors draw a highly generalizing conclusion without any differentiation, for example, by disorders or degree of risk.

• For their ratings, Hofmann et al. [2] did not report basic data on the number and training of raters, on blinding, or interrater-reliability. For the best minimization of bias, raters of both approaches would have been included as done by Gerber et al. and Thoma et al. [3, 4]. Thus, the quality of the procedures applied by Hofmann et al. [2] themselves is questionable.

• The conclusions by Hofmann et al. [2] are based mostly on “unclear” designations, not clear flaws. In fact, an “unclear” risk of bias indicates that the design feature could be both worse or better than described in the article. What is most concerning is that the authors did not make any effort to resolve the “unclear” assignments by carefully reading the papers or contacting their authors. Many assignments are obviously clear from the studies [e.g. 5, 6].

• In addition, even if there are flaws, Hofmann has not shown that these particular flaws lead to results in favor of PDT (rather than e.g., greater error in effect estimates overall). Several meta-analyses did neither find significant correlations between ratings of methodological quality and outcome [4, 7] nor between treatment integrity (assessed by prominent CBT researchers, e.g. Aaron T. Beck) and differences in outcome between CBT and PDT [8].

• If the "poor quality" of PDT studies "invalidates" [2] the results reported by Leichsenring et al. [1] making them "meaningless", this would equally apply to meta-analyses carried out by CBT researchers who included several of these same studies. Tolin [9], for example, included 10 studies also included by Leichsenring et al. [1]. Hofmann, however, has never critically commented on these meta-analyses - which are interpreted as supporting the efficacy of CBT - thus, again applying a double standard.

• Hofmann was repeatedly shown to apply double standards when judging studies of CBT vs. PDT [10, p. 49-51].

(a) In a previous meta-analysis, for example, Hofmann [11, p. 180] claimed that the quality of studies included by him was "considerably better" than that of studies of PDT, e.g. in the meta-analysis by Leichsenring and Rabung [12]. Hofmann et al. [11] reported a mean Jadad score of 1.23, whereas the mean Jadad score in the meta-analysis by Leichsenring et al.[12] was 1.96 (0 = poor; 5 = excellent).

(b) Hofmann [11]criticized the meta-analysis by Leichsenring et al. [12] for including heterogeneous studies. However, between-effect hterogeneity was in the low to medium range [10, 12]. In his own meta-analysis, Hofmann [11] did not even test for heterogeneity before combining data of randomized controlled and observational studies [10, 11].

• Thus, from a scientific perspective, it is questionable whether a strong proponent of CBT who has publicly demonstrated that he is an opponent of PDT [e.g. 13] is able to provide unbiased conclusions about PDT.

Given the author’s very negative publicly expressed opinions about PDT, and the way he conducted this critique of the Leichsenring et al. review [1] it appears that biases can lead to a lack of even-handedness in regard to the evaluation of psychodynamic studies. Thus, we respectfully would ask that if he chooses to write about PDT(e.g., comment on a meta-analysis, conduct a meta-analysis, or conduct a study involving PDT), that he involve a psychodynamic researcher in the process (i.e., implement a version of adversarial collaboration [14], and also that he recuse himself from being involved as an editor, or reviewer, in regard to research involving PDT.

We would welcome the collaboration of CBT researchers in researching psychotherapy and synthesizing the results from trials. We have done so several times [e.g. 15, 16].

Given the present crisis of replicability of research [17], biased and tendentious statements as those by Hofmann bear the risk of damaging all psychotherapy research equally in the eyes of the public.

References

1 Leichsenring F et al.: Psychodynamic therapy meets evidence-based medicine: a systematic review using updated criteria. Lancet Psychiatry 2015;2:648-660. 2 Hofmann SG et al.: Comment from PubMed Commons. January 23rd, 12:28am UTC 2016 3 Thoma NC et al.: A quality-based review of randomized controlled trials of cognitive-behavioral therapy for depression: an assessment and metaregression. Am J Psychiatry 2012;169 4 Gerber AJ et al.: A quality-based review of randomized controlled trials of psychodynamic psychotherapy. Am J Psychiatry 2011;168:19-28. 5 Barber J et al.: Short-term dynamic psychotherapy versus pharmacotherapy for major depressive disorder: a randomized, placebo-controlled trial. J Clin Psychiatry 2012;73:66-73. 6 Crits-Christoph P et al.: Psychosocial treatments for cocaine dependence: National Institute on Drug Abuse Collaborative Cocaine Treatment Study. Arch Gen Psychiatry 1999;56:493-502. 7 Keefe JR et al.: A meta-analytic review of psychodynamic therapies for anxiety disorders. Clin Psychol Rev 2014;34:309-323. 8 Leichsenring F et al.: Treatment integrity: An unresolved issue in psychotherapy research. Curr Psych Rev 2011;7 313-321. 9 Tolin DF: Is cognitive-behavioral therapy more effective than other therapies? A meta-analytic review. Clin Psychol Rev 2010;30:710-720. 10 Leichsenring F et al.: Double standards in psychotherapy research. Psychother Psychosom 2011;80:48-51. 11 Hofmann SG et al.: The effect of mindfulness-based therapy on anxiety and depression: a meta-analytic review J Consult Clin Psychol 2010; 78 169-183. 12 Leichsenring F et al.: Effectiveness of long-term psychodynamic psychotherapy: a meta-analysis. JAMA 2008;300:1551-1565. 13 Rief W et al.: [Saving psychoanalysis. At any cost?]. Nervenarzt 2009;80:593-597. 14 Mellers B. et al.: Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychol Sc 2001;12:269-275. 15 Leichsenring F et al.: The efficacy of short-term psychodynamic psychotherapy in specific psychiatric disorders: a meta-analysis. Arch Gen Psychiatry 2004;61:1208-1216. 16 Leichsenring F et al.: Psychodynamic therapy and CBT in social anxiety disorder: a multicenter randomized controlled trial. Am J Psychiatry 2013;170:759-767. 17 Open Science Collaboration: PSYCHOLOGY. Estimating the reproducibility of psychological science. Science 2015;349:aac4716.

On 2015 Oct 05, Lydia Maniatis commented:

It's been a month since I posted my comments on this article and on the original Blakeslee and McCourt article. A sharp increase in visits to the target articles post-comment seems to indicate that the comments are being read, but no author or reader has put in their two cents, either here or on PubPeer. I think that my argument against the "brightness-is-perceived-luminance" idea is sound and straightforward. Am I wrong?

On 2015 Sep 06, Lydia Maniatis commented:

I fully agree with the latter part of this commentary but disagree with the author's concession that, in lightness experiments, "when illumination appeared homogeneous, lightness and brightness judgments were identical. There is nothing new here. It is well known."

Brightness is currently being described, including by Gilchrist, as the perceptual correlate of luminance. But there is no perceptual correlate of luminance, even under (apparently) homogeneous illumination, and this can be proved as follows:

We ask an observer to report on the lightness of a set of surfaces which don't produce the impression of shadows or transparency. Then, in a second session, we present the same set of surfaces under a different level of illumination. The lightness reports for the surfaces will stay essentially the same, even though their luminances may have changed substantially. So to say that people are making "brightness" judgments in either the first or the second or in any case doesn't seem reasonable.

On 2018 Jan 24, Doug Berger commented:

Excellent article from a brave young researcher willing to put her career in the line of fire of criticism of CBT and other psychotherapy adherents, not to mention employment opportunities where psychotherapy research has been a mainstay for years and is a cash-cow. The only caveat I would add is that the translation of basic science research into the ability og CBT to overcome the inability to single-(patient blind), or double-(patient and therapist blind), as well as the inability to have blind placebo, in a psychotherapy clinical trial is a serious doubt. Many psychiatric study drugs over the years have had considerable pre-clinical basic science data, even phase I or phase II clinical data, only to fail miserably in phase III trials when adequate numbers of subjects were studied in tightly blinded conditions with blind placebo control. Basic science research only lends itself to decide if it is worth while to test a drug in large populations clinically, it doesn’t say itself if a drug will work, especially in psychiatric conditions where endpoints are subjective and random error high.

The whole premise of CBT, that negative or distorted cognitions are the cause of a psychiatric condition is the only instance in all of medicine and psychiatry where a symptom of an illness is also construed to be the cause. Taking a symptom and making it into a cause is a way to spin the need of a therapy aimed at a cause that actually the result, and in psychiatric conditions with subjective endpoint hope and expectation effects are an easy way to garner some symptom reduction that can easily be called a "response" in a clinical trial. Reading the paper, I wonder if the journal asked the authors to propose ways to work around the blinding problem. The work-around is heuristic but not of practical value once a clinical trial gets going-and as the authors rightly note (and as I have noted in other papers below), there is really no way to do blind/blind placebo psychotherapy clinical trial.

Other papers on this topic: https://www.ncbi.nlm.nih.gov/pubmed/26870318 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4863672/ https://www.japanpsychiatrist.com/Abstracts/CBT_Escape.html

D. Berger, U.S. Board Certified Psychiatrist

On 2015 Aug 25, Ranjith Ramasamy commented:

https://www.youtube.com/watch?v=qAaSG4LDSE4&feature=youtu.be - Video from Urology times summarizing the study findings

On 2015 Aug 23, Kelly Drew commented:

Hello to all! Interesting paper, but don't dismiss A1AR. Have you seen these papers?

1: Muzzi M, Coppi E, Pugliese AM, Chiarugi A. Anticonvulsant effect of AMP by direct activation of adenosine A1 receptor. Exp Neurol. 2013 Dec;250:189-93. doi: 10.1016/j.expneurol.2013.09.010. Epub 2013 Sep 19. PubMed PMID: 24056265.

2: Muzzi M, Blasi F, Masi A, Coppi E, Traini C, Felici R, Pittelli M, Cavone L, Pugliese AM, Moroni F, Chiarugi A. Neurological basis of AMP-dependent thermoregulation and its relevance to central and peripheral hyperthermia. J Cereb Blood Flow Metab. 2013 Feb;33(2):183-90. doi: 10.1038/jcbfm.2012.157. Epub 2012 Oct 24. PubMed PMID: 23093068; PubMed Central PMCID: PMC3564191.

3: Muzzi M, Blasi F, Chiarugi A. AMP-dependent hypothermia affords protection from ischemic brain injury. J Cereb Blood Flow Metab. 2013 Feb;33(2):171-4. doi: 10.1038/jcbfm.2012.181. Epub 2012 Dec 5. PubMed PMID: 23211965; PubMed Central PMCID: PMC3564206.

On 2015 Aug 26, Jim Woodgett commented:

I sound like a broken record, but it is quite remarkable that yet another paper presumes an inhibitor is specific for GSK-3beta. In this case (AR-A014418) is equally as effective at inhibiting GSK-3alpha, as is every other small molecule inhibitor of GSK-3, including lithium. This is a problem because the authors fail to assess the contribution of the second isoform which is being equivalently blocked in the experiments. There are numerous papers that describe the common and distinctive functions of these protein kinases.

On 2016 Feb 08, Salzman Lab Journal Club commented:

This paper presents a striking view of the intron lariat spliceosome at unprecedented resolution and uses pombe as a strategy for enriching relatively homogenous structural complexes. As our lab is interested in circular RNA, we were curious about how this structure may be used to model minimal circularized exon lengths. We look forward to more structures in the future that give a more detailed look at the exon-exon junction!

On 2016 Nov 03, David Juurlink commented:

Post-publication note: Additional analysis of absolute risk of opioid-related death among patients receiving more than 200 mg morphine (or equivalent) per day reported here http://bit.ly/2fVaQYL

On 2015 Aug 22, Arnaud Chiolero MD PhD commented:

An enlightening commentary on the necessity of revisiting DCIS treatment. It also raises question about screening - the major driver of DCIS identification.

On 2015 Nov 26, Tero Kivelä commented:

This paper is commented in http://www.ncbi.nlm.nih.gov/pubmed/26288835

On 2015 Nov 26, Tero Kivelä commented:

This is a commentary on http://www.ncbi.nlm.nih.gov/pubmed/26288849

On 2015 Aug 24, Andrew R Kniss commented:

This “Perspective” piece is basically a plea from Dr. Landrigan and Dr. Benbrook for “all aspects of the safety of biotechnology” to be “thoroughly reconsider[ed]”. However, in the two page opinion, they provide no evidence that crop biotechnology is harmful. In fact, Landrigan and Benbrook acknowledge that the National Academy of Sciences (NAS) “has twice reviewed the safety of GM crops” and they do not dispute the scientific consensus expressed by NAS that “GM crops pose no unique hazards to human health.” The way I read it, the entire Perspective piece seems to be a muddled conflation of two separate (albeit related) issues; the use of GMO crops and the use of herbicides. A full critique here: http://goo.gl/IcZt2S

Dr. Landrigan and Dr. Benbrook cite glyphosate-resistant weeds as a primary reason why “fields must be now be treated with multiple herbicides,” but this point also deserves some scrutiny. In corn, for example, multiple herbicides have been a common practice since long before GMO crops were introduced. In the year 2000, before Roundup Ready GMO corn had gained widespread adoption in the US (and also before glyphosate-resistant weeds were growing in corn fields), corn growers were applying nearly three herbicide active ingredients per acre. The latest USDA data from 2014 show fewer than 3.5 active ingredients applied per acre. This suggests that while glyphosate-resistant weeds may certainly have increased the number of herbicides used per acre compared to 5 years ago, the change has been relatively modest when compared to herbicide use before adoption of GMO crops.

Another misleading statement made by Dr. Landrigan and Dr. Benbrook is that the “risk assessment gave little consideration to potential health effects in infants and children, thus contravening federal pesticide law.” This claim was also addressed explicitly by EPA in their FAQ document (http://www2.epa.gov/ingredients-used-pesticide-products/registration-enlist-duo). EPA concluded that after incorporating a 10X safety factor for children, and based on a “complete and very robust” data set, that the “risks were still acceptable for all age groups for all components of the assessment: dietary food and drinking water exposure, volatility, spray drift, residential, and aggregate assessment.” This claim is addressed in even more detail (600 words) in the EPA’s response to public comment (http://www.regulations.gov/contentStreamer?documentId=EPA-HQ-OPP-2014-0195-2414&disposition=attachment&contentType=msw8).

On 2015 Sep 21, M Mangan commented:

It should be further noted that although disclosure documents were updated for Dr. Benbrook in August, following the publication of an article in the NYTimes in September, new issues of potential conflicts of interest were unearthed. Documents provided from a FOIA request by Eric Lipton illustrated that both Benbrook and Landrigan were actively working with the "Just Label It" organization and Gary Hirshberg of the organic food industry, during the time of preparation of this work. http://www.nytimes.com/interactive/2015/09/06/us/document-benbrook.html

My request to the NEJM to investigate this and update the disclosure documents, due to the new evidence, was declined by the NEJM.

On 2015 Aug 20, M Mangan commented:

This piece is astonishingly flawed on several levels, but most importantly has basic facts that are incorrect or blatantly misrepresented. Please see some capable assessments of the claims here, from a researcher who studies herbicides: http://weedcontrolfreaks.com/2015/08/gmos-herbicides-and-the-new-england-journal-of-medicine/

I would also encourage readers to seek out the expert reactions from the Science Media Centre: http://www.sciencemediacentre.org/expert-reaction-to-gmos-herbicides-and-public-health/

But I would also like to note that the calls to provide information via labels do not indicate what label they think would apply to their herbicide concerns. None of the proposed labels in the US, or any other country I've seen, address herbicides in any manner. Since non-GMOs also use herbicides, and some GMOs do not, calling for a misleading label would be irresponsible for health professionals.

On 2015 Sep 16, F Morceau commented:

Interesting article but did the authors assessed the effect of BML-210 on NB4 cells differentiation? It woud have been interesting and relevant to provide this information. ATRA is mentioned in the materials and methods section while it has not been used in fact in the article! How this could escape to the reviewers?

On 2017 May 30, Rashmi Das commented:

We thank Harri for his PERSONAL (NON PEER REVIEWED) OPINION which is available at above HANDLE ( http://hdl.handle.net/10138/153180) THAT CONTAINS DIRECT COPY AND PASTE OF THREE FIGURES/IMAGES FROM OUR PREVIOUS PUBLICATIONS (JAMA 2014 and Cochrane 2013). We are happy to reply to above comments made by Harri. First regarding the Cochrane review which was withdrawn in 2015, the detailed report is already available at following link (http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD001364.pub5/abstract). This report is the collaborative observation and conclusion of the Cochrane editors (UNLIKE THE HANDLE WHICH CONTAINS MORE OF PERSONAL OPINION WHICH HAS ALREADY BEEN EXAMINED BY THE COCHRANE EDITORS BEFORE REACHING THE CONCLUSION). The same HANDLE WAS SENT TO JAMA EDITORS REGARDING THE JAMA CLINICAL SYNOPSIS (PUBLISHED IN 2014) AND HARRI REQUESTED THE EDITORS TO CARRY OUT THE INVESTIGATION AND VERIFY. THE EDITORS ASKED US FOR REPLY WHICH WE CLARIFIED IN A POINT TO POINT MANNER (BOTH THE COMMENT BY HARRI AND OUR REPLY WAS PUBLISHED, SEE BELOW). HAD THE COMMENT/REPORT BY HARRI WAS ENTIRELY CORRECT, THE JAMA EDITORS COULD HAVE STRAIGHTWAY RETRACTED/WITHDRAWN THE SYNOPSIS WITHOUT GOING FOR PUBLICATION OF THE COMMENT/REPLY (Both are available at following: https://www.ncbi.nlm.nih.gov/pubmed/26284729; https://www.ncbi.nlm.nih.gov/pubmed/26284728). IT HAS TO BE MADE CLEAR THAT THE JAMA SYNOPSIS (DAS 2014) WAS WITHDRAWN AS THE SOURCE DOCUMENT ON WHICH IT WAS BASED (COCHRANE 2013 REVIEW) WAS WITHDRAWN (NOT BASED ON THE REPORT IN THE HANDLE WHICH IS A PERSONAL NON PEER REVIEWED OPINION). The irony is that though HARRI'S COMMENT got published as LETTER TO EDITOR in JAMA after OUR REPLY, still the NON PEER REVIEWED HANDLE THAT CONTAINS DIRECT COPY OF THREE FIGURES/IMAGES FROM OUR PUBLICATION IS GETTING PROPAGATED.

On 2015 Aug 30, Harri Hemila commented:

This is a short comment on Das RR, 2014. A much more detailed description of problems in Das RR, 2014 is available at HDL.

The paper by Das RR, 2014 was based on their Cochrane review Singh M, 2013, which had a number of problems which are descibed in HDL.

The Cochrane review was withdrawn in April 2015, see DOI.

On 2016 Feb 04, Hamid Salehiniya commented:

I read the article,this is a attractive and useful article, based on previous research, we conclude obesity is a important risk factor for cancer, in conclusion one strategy in cancer prevention is active life style.

On 2015 Aug 28, Lydia Maniatis commented:

There seems to be a chicken and egg problem going on here as well. Here: https://www.youtube.com/watch?v=QORWM3Pl760, one of the authors seems to be defining "stimulus" as something that is constructed by the visual system, as opposed to a light -reflecting object in the world, and at the same time he is saying that the nature of this stimulus is determined in some way by its frequency of occurrence. But the latter has no existence prior to the former. There has to be some reference at some point to an objective situation, a stimulus in that sense, for the conversation to work.

Do the authors consider there to be a link between the topography of the light energy on the retina and the topography of the light energy in the world one moment before it strikes the retina?

On 2015 Aug 28, Lydia Maniatis commented:

The claims in this article (and related past publications) are not credible, and barely defended.

First, there is a little bit of sleight-of-hand in the authors' definition of “wholly empirical,” with the result that it includes both wholly acceptable and wholly unacceptable assertions. After describing their view that the visual system assigns perceptual values “without ever recovering or statistically estimating the properties of objects and conditions in the visual environment,” they label this view “wholly empirical” on the basis that it “depends entirely on feedback from trial and error experience.” But unlike the former claim, which is highly disputable, the latter seems simply to be a description of evolution by natural selection, a trial and error process in which adaptive features are preserved and less adaptive ones rejected. I cannot imagine that any scientist today would propose a mechanism for a biological function that they do not believe could have arisen via natural selection. Thus, it does not seem fair on the part of the authors to monopolize the concept for a specific set of proposals.

They reinforce this apparent monopoly by implying that a lack of correlation between the percept and the physical world is a necessary consequence of the link between perceptual processes and behavior: “...the biological feedback loop...would progressively order the basic visual qualities we perceive (apparent length, lightness, etc) according to their impact on biological (reproductive) success, rather than with the generative properties in the world (actual physical lengths, the reflective and illumination values underlying luminance, etc) [On what basis do the authors refer to reflectance and illumination as objective properties of the world (see below)?]” The “rather than” term is a verbal trick, implying that a perceptual representation that is correlated with (selected) physical properties would be less adaptive than one that is not. But such an assumption is not warranted.

The “evidence” for the position is also contingent on looseness in the supporting arguments. We are told, for example, that “since luminance measures the number of photons falling on the retina, common sense suggests that measurements of light intensity and perceived lightness should be proportional. Thus, if two surfaces return the same amount of light to the eye, they “should” be perceived as equally light.” This is a straw man. In the most extreme cases of same-luminance surfaces producing different lightness, the different surfaces, in addition to producing lightness percepts, also produce illumination percepts. Higher perceived lightness correlates with lower perceived illumination. The visual system is attempting to provide relative estimates for reflectance and illumination. This could not be achieved by directly representing luminance, whatever “common sense” might say. It is not correct to say that the visual system never correctly (or to a good approximations) represents relative reflectance/illumination values across surfaces. When it does, it is not by accident, it is what the system is designed (so to speak) to achieve. The idea that all images that elicit reflectance/illumination percepts do so on the basis of the frequency of evolutionary experience - behavioral responses to each particular stimulus, and their consequences for the reproductive success of the individual - is not credible. Even if we take apparent illumination out of the picture, the problem is just as big. Are we supposed to explain e.g. every Kanisza figure, on the basis of how frequently it occurred?

And, for that matter, the notion that “sensory stimuli [are] associated with biologically useful responses” does not justify the claim that stimulus frequency determines perceptual values, since a stimulus pattern may be very frequent but have little biological consequence, or rare but have a larger biological impact. So, in another sleight-of-hand, the authors have slipped an unwarranted frequency-biologically-relevant link into their argument.

In addition to the lightness example, the other piece of “evidence” offered is equally problematic on a number of counts. First, we are told that psychophysical experiments indicate that lines oriented at about thirty degrees from the vertical appear longer than lines at any other orientation. But the perceived length of a line is entirely contingent on the figure in which it is incorporated. So even if the authors claim that, over evolutionary time, line segments with the thirty degree orientation occurred (and were reacted to in an evolutionary dispositive way), more often than any other orientation, this would not explain why an oblique with a greater deviation from the vertical will reliably yield a longer percept than a vertical when incorporated in, e.g. a Shepard box. Even if we take the authors' natural scene statistics at face value, they are not relevant, since edges always occur in objects, and the shapes of these objects mediate the perception of length of an edge. Assumptions regarding the shapes of objects even determine whether or not a physically present or absent edge will or will not be seen.

The natural scene statistics offered are also not credible. Human ancestors varied differed greatly in size form the present, individual humans differ in height based on age, the eyes and body are constantly in motion, the distance to the object being viewed is constantly changing, and all of these things affect the orientation of the projection of a physical edge. Most of our time is arguably spent looking at close range, and at people. A statistical distribution developed on the basis of measurements from a fixed height at a relatively great distance of groomed gardens is arguably not a good approximation of human experience. Finally, are the authors really claiming that an isolated line segment of thirty degrees looks longer than others because it (supposedly) projected more frequently on the retinas of humans and their ancestors? If all percepts are drawn from frequency distributions, then how do we account for their qualitative differences? Are colors also labelled on the basis of the frequency that each collection of wavelengths occurs?

There is, finally, a fundamental contradiction in the argument that perceptual values don't ever recover or estimate properties of objects and conditions in the environment (I discuss this contradiction in Maniatis (in press). The problem is that if we choose to adopt this view, then we are not entitled to refer to the physical world as though we did, in fact, have knowledge about it. Thus, statements such as “consider the objective length of a line, e.g. the edge of a ruler...” are paradoxical because they imply that the authors have access to the very objective facts that they claim are inaccessible. It does no good to argue that we detect objective facts using objects called instruments, since the properties of these objects, too, are only accessible through perception. Given their position, the authors are not entitled to make any reference to the objective world and its properties, since such references directly contradict it.

On 2015 Aug 26, Lydia Maniatis commented:

In a search of the book "Perceiving Geometry," the term 'shape' comes up three times, twice in reference to the shape of distributions and once in a plain assertion that some neurons respond selectively to "higher-order stimulus characteristics...such as shape...". The assertion is not accompanied by arguments. The book has many references, are there any in particular that address the issue of shape?

The term or concept of 'figure-ground' also does not arise in the book. Do you consider the problems of perceptual organisation to have been solved on the basis of frequency distributions?

On 2015 Aug 24, Dale Purves commented:

Shape is a higher order construct of "size, distance, orientation" and geometrical factors such as the length of intervals (lines) and angles. See Howe and Purves "Perceiving Geometry" (Springer, 2005) for a full account and references to original papers.

On 2015 Aug 24, Lydia Maniatis commented:

In their summary, the authors state that:

"Visual perception is characterized by the basic qualities of lightness, brightness, color, size, distance, orientation, speed and direction of motion ordered over some range."

They have left out the most basic quality of all, and the one that largely mediates all the others: Shape. Was this an oversight, or a purposeful or principled omission?

They also say that: "These perceived qualities and their order within these ranges, however, do not align with reality."

They should clarify whether they mean for this second statement to apply also to shape.

On 2016 Jan 14, Arturo Casadevall commented:

The central criticism is that we have compared variables for which there is no causal relationship. We recognize the difficulties involved in assuming causality and dangers of spurious correlations when plotting unrelated variables. Furthermore, we are fully aware that correlation is not causation. However, the criticism made by Levine and Weinstein does not take into account a large body of published scholarly work showing that spending of public funds translates into medical goods such as new therapeutics. To make this point, we note the findings of several studies. In 2000, a United States Senate Report found that of the 21 most important drugs introduced between 1965-1992, 15 (71%) ‘were developed using knowledge and techniques from federally funded research’ (http://www.faseb.org/portals/2/pdfs/opa/2008/nih_research_benefits.pdf). A recent study of 26 transformative drugs or drug classes found that for many, their discovery was made with governmental support [1]. Numerous other studies have reinforced this point [2-4]. Kohout-Blume estimated that a 10% increase in targeted funding for specific diseases produced a 4.5% increase in the number of drugs reaching clinical trials after an average lag of 12 years [5]. In our own investigations, we have traced the ancestry of most of the drugs licensed in the past four decades to publicly funded research (unpublished data). The literature in this field overwhelmingly supports the notion that public spending in biomedical research translates into public goods. The debate is not about whether this happens but rather about the magnitude of the effect. The notion that public funding in biomedical research generates basic knowledge that is subsequently used in drug development is a concept accepted by most authorities. Hence, the use of the NIH budget as a proxy for public spending in biomedical research is appropriate.

We are aware that establishing causal relationships among non-experimental variables can be a daunting task. However, we note that the relationship between public spending and medical goods does meet some of the essential criteria needed to establish causality. First, the relationship between these variables meets the requirement of temporal causality since, for many drugs, publicly funded basic research precedes drug development. Second, we also have mechanistic causality, since knowledge from basic research is used in designing drugs. There are numerous examples of mechanistic causality including the finding of receptors with public funds that are then exploited in drug development when industry generates an agonist or inhibitor. We acknowledge that we do not know if the relationship between public spending and drug development is linear, and the precise mathematical formulation for how public spending translates into medical goods is unknown. In the absence of evidence for a more complex relationship, a linear relationship is a reasonable first approximation, and we note that other authorities have also assumed linear relationships in analyzing inputs and outcomes in the pharmaceutical industry. For example, Scannell et al. [6] used a similar analysis to make the point that ‘The number of new drugs approved by the US Food and Drug Administration (FDA) per billion US dollars (inflation-adjusted) spent on research and development (R&D) has halved roughly every 9 years’.

The authors claim to have done a causality analysis of the data generated in our paper, concluding that ‘We do not find evidence that NIH budget ⇒ NME (p=0.475), and thus it may not be a good indicator of biomedical research efficiency.’ However, this oversimplifies a very complex process of how public spending affects NME development; we do not agree that this simple analysis can be used to deny causality. Although the limited information provided in their comment does not permit a detailed rebuttal, we note that a failure to reject the Granger causality null hypothesis does not necessarily indicate the absence of causality. Furthermore, Granger causality refers to the ability of one variable to improve predictions of the future values of a second variable, which is distinct from the philosophical definition of causality. Whether or not NIH budget history adds predictive ability in determining the number of NMEs approved at some point in the future cannot negate the fact that basic biomedical research funding unequivocally influences the creation of future drugs, as well as many other outcomes. Therefore, we stand by our use of NIH funding and NMEs as indicators of biomedical research inputs and outcomes.

The authors suggest that another study by Rzhetsky et al. [7] contradicts the findings of our paper and provides a better method of studying biomedical research efficiency. The work by Rzhetsky et al., while very interesting, addresses a fundamentally different question relating to how scientists can most efficiently choose research topics to explore a knowledge network [7]. The allocation of scientists to research topics is undoubtedly a possible contributor to overall research efficiency, but the approach used in this analysis is very different from our broader analysis of the biomedical research enterprise as a whole. The work in [7] has a narrow scope and does not attempt to study the impact of research investments in producing societal outcomes. The central conclusion of our paper is that biomedical research inputs and outputs are increasing much faster than outcomes, as measured by NMEs and LE.

We do not ‘conjecture that a lack relevance or rigor in biomedical research’ is solely responsible for this phenomenon, as Levine and Weinstein assert. Instead, our paper discusses a number of possible explanations—many of which have been previously identified in the literature [6-12], including several that agree with the conclusions of Rzhetsky et al. [7]. However, the recent epidemic of retracted papers along with growing concerns about the reproducibility of biomedical studies, expressed in part by pharmaceutical companies dedicated to the discovery of NMEs [13, 14], are indisputable facts. If a substantial portion of basic science findings are unreliable, this is likely to contribute to reduced productivity of the research enterprise. We agree with the suggestion that research difficulty increases as a field matures, which has been made by others [6]; this does not contradict our analysis and is mentioned in our paper’s discussion. Biomedical research efficiency is complex, and it is likely that the decline in scientific outputs has numerous causes. It is appropriate for scientists to consider any factors that may be contributing to this trend, and the comments from Dr. Schuck-Paim in this regard (see the other posted comments) are therefore welcome.

In summary, we do not find the arguments of Levine and Weinstein to be compelling. We note that other investigators have come to conclusions similar to ours [6, 15]. The productivity crisis in new drug development has been intensively discussed for at least a decade [6, 15-16]. We believe that addressing inefficiencies in biomedical research is essential to maintain public confidence in science and, by extension, public funding for basic research.

Arturo Casadevall and Anthony Bowen

[1] Health Aff (Millwood ) 2015, 34:286-293.

[2] PNAS 1996, 93:12725-12730.

[3] Am J Ther 2002, 9:543-555.

[4] Drug Discov Today 2015, 20:1182-1187.

[5] J Policy Anal Manage 2012, 31:641-660.

[6] Nat Rev Drug Discov 2012, 11:191-200.

[7] PNAS 2015, 112:14569-14574.

[8] Res Policy 2014, 43(1):21–31.

[9] Nature 2015, 521(7552):270–271.

[10] Br J Cancer 2014, 111(6):1021–1046.

[11] Nature 2015, 521(7552):274–276.

[12] J Psychosom Res 2015, 78(1):7–11.

[13] Nature 2012, 483(7391):531-533.

[14] Nat Rev Drug Discov 2011, 10(9):712.

[15] Nat Rev Drug Discov 2009, 8:959-968.

[16] Innovation policy and the economy 2006, 7:1-32.

On 2016 Jan 08, Michael LeVine commented:

In this recent article, Bowen and Casadevall attempt to classify biomedical research efficiency in terms of the ratio of outcomes to input. The outcomes were chosen to be approved new molecule entities (NMEs) and US life expectancy (LE); the chosen input was the NIH budget. While the resulting analysis claims that efficiency has decreased in the last decade, we argue that (i)-the analysis performed is insufficient to make that claim, and (ii)- the findings do not support the conjecture that a lack of relevance or rigor in biomedical research is causing stagnation in medicine and public health.

Bowen and Casadevall suggest that because research projects take time to complete, it is possible that “the exponential growth in research investment and scientific knowledge over the previous five decades has simply not yet grown fruit and that a deluge of medical cures are right around the corner”. They investigate time-lagged efficiency for NMEs, but this analysis is only sufficient if there is a linear causal relationship between the two variables that is unaffected by any external variables that have not been included in the analysis. Without any evidence of such a relationship, it is unwise to interpret a trend in a ratio between two unassociated measurements. Just as two unrelated measurements can display a spurious correlation, the ratio between those measurements may display a spurious trend.

We reanalyzed the data used in this paper to find evidence of causal relationships between the inputs and outcomes. To do this, we tested for Granger causality (1), which identifies potentially causal relationships by determining whether the time series of one variable is able to improve the forecasting of another. We analyzed the non-stationary time series from 1965-2012 using the Toda and Yamamoto method (2), which utilizes vector autoregression. We will refer to a variable X improving the forecasting of a variable Y as X ⇒ Y.

We do not find evidence that NIH budget ⇒ NME (p=0.475), and thus it may not be a good indicator of biomedical research efficiency. However, we do find evidence that NIH budget ⇒ LE (p<10-8) and NIH budget ⇒ publications (p<machine precision). Notably, however, both VAR models utilize the maximum possible time lags (15, as selected using the Akaike information criterion (3), coincidentally the same number as used in this paper), and do not pass the Breusch-Godfrey test (4) for serially correlated residuals. As this suggests that more time lags are required to build appropriately rigorous models, it seems unwise to over-interpret the potential Granger causal relationships or make any comparisons between the Granger causality during different periods of time, until significantly more time points are available. Even with additional data, the serial correlation in the residuals might not be alleviated without the use of more complex models including non-linear terms or external variables. All three of these possible limitations also affect the analysis in this paper, but no statistical tests were performed there to assess the robustness of results.

We conclude that this published study of biomedical research efficiency is insufficient methodologically because models of greater complexity are required. From our reanalysis, we are not able to support the hypothesis that biomedical research efficiency is decreasing. Instead, we can only conclude that from 1965-2012, the NIH budget may have a causal effect on LE and publication, but that more time points are required to improve the models.

Another recent work (5), which aimed to study the scientific process and its efficiency, also suggested that the efficiency of biomedicinal chemistry research (defined in terms of the number of experiments that would need to be performed to discover a given fraction of all knowledge) has decreased over time. However, the analysis in (5) also suggested that even the optimal research strategy would eventually display a decrease in efficiency, due to the intrinsic increase in the difficulty of discovery as a field matures. While there are additional limitations and assumptions involved in this analysis, (5) provides an example of the level of complexity and quantitative rigor required to study research efficiency, and implies an alternative explanation for the potential reduction in biomedical research efficiency. Considering the findings in (5), we deem the hypotheses proposed in this paper, which suggests that a lack of relevance or rigor in biomedical research is causing stagnation in medicine and public health, to be unfounded.

We write this comment in part because unfounded defamatory claims directed at the scientific community are dangerous in that they may negatively affect the future of scientific funding. Such claims should not be made lightly, and the principle of parsimony should be invoked when less defamatory alternative hypotheses are available.

Michael V. LeVine and Harel Weinstein

(1) Granger CWJ (1969) Investigating Causal Relations by Econometric Models and Cross-spectral Methods. Econometrica 37(3):424–438.

(2) Toda HY, Yamamotob T (1995) Statistical inference in vector autoregressions with possibly integrated processes. J Econom 66:225–250.

(3) Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723.

(4) Breusch TS (1978) Testing for autocorrelation in dynamic linear models. Aust Econ Pap 17(31):334–355.

(5) Rzhetsky A, Foster JG, Foster IT, Evans JA (2015) Choosing experiments to accelerate collective discovery. Proc Natl Acad Sci:201509757.

On 2016 Jan 14, Arturo Casadevall commented:

We appreciate the comment by Dr. Schuck-Paim and we fully agree that increased transparency and reporting of data during the process of therapeutic development can only benefit the scientific enterprise and enable a more efficient use of limited resources. Some of the other mentioned issues, including relevance of animal models, underpowered study design, and errors during data analysis and reporting, have all been implicated in the literature as referenced by Dr. Schuck-Paim and cited in our paper. We agree that each of these issues merits attention and expect that new tools will need to be developed to address some problems. One example would be the development of drug screening chips containing human cells as an alternative to some animal models, which may have poor predictability of a drug’s toxicity in humans (http://www.ncats.nih.gov/tissuechip).

Anthony Bowen

Arturo Casadevall

On 2015 Nov 05, Cynthia Schuck-Paim commented:

In exploring progress in biomedical research, the authors show that human life expectancy and the number of new molecular entities (NME) approved by the FDA have remained relatively constant over the last decades, despite increasing financial input, research efforts and publication numbers.

To explain the slowing of therapeutic innovation they consider several negative pressures acting on the field, including prior resolution of simpler research problems, increasing regulation, overreliance on reductionist approaches (including use of animal models), and the poor quality of published research. The high prevalence of irreproducible results, obscure methods, poorly designed research and publication biases are also mentioned.

Many of these issues would greatly benefit from initiatives that promote transparency at the various stages of the therapeutic development pipeline. It has been widely acknowledged that poor reporting prevents the accurate assessment of drug and intervention efficacy. Indeed, pre-clinical research and in vivo methods have been shown to be particularly prone to biases and selective reporting of outcomes, leading to bad decision-making, wasted resources, unnecessary replication of efforts and missed opportunities for the development of effective drugs (1). One proposal to address this issue is the extension of good disclosure practice to the pre-clinical phase by conditioning publication to the registration of the pre-clinical trial prior to the commencement of the study (2). Indeed, the exact same reasons that compelled prospective registration and deposition of clinical trial results in public databases apply to preclinical studies.

Still, no matter how transparent, well-designed, -analyzed and -reported the research, results generated from inappropriate models will not be successfully translated into valid disease contexts. Currently, most pre-clinical studies are based on the use of animal models, despite the increasing number of articles showing that they are an expensive and ineffective option to explore pathophysiological mechanisms, evaluate therapeutics, and decide on whether drug candidates should be carried forward into the clinical phase (3-5).

Failure rates in the clinical phase are around 95% (6), mainly due to the limited power of animal studies to predict NME efficacy, safety and toxicity in humans. These predictive odds vary depending on the understanding and complexity of disease biology: while for therapeutics targeting infectious diseases success rates are higher, for diseases involving complex mechanisms, such as cancer, they can be as low as 2.3% (7). Such low predictability drains the entire system by funneling limited resources into outputs that often fail.

In addition, false negatives at the pre-clinical stage eliminate a large part of NMEs that may have succeeded otherwise. Let us not forget Aspirin, a blockbuster drug that would not make the preclinical trial phase if tested today given its unacceptably high toxicity in animal tests. Animal-based pre-clinical phases are certainly pivotal in explaining the small number of NMEs identified in the last decades. Implementation of methods that are more faithful to the human biology is crucial for the much needed progress to ameliorate human disease and suffering.

References

(1) Macleod MR, 2015 Risk of bias in reports of in vivo research: a focus for improvement. PLoS Biol 13: e1002273

(2) Kimmelman J, Anderson JA (2012). Should preclinical studies be registered? Nat Biotechnol 30: 488–489

(3) Hartung T (2013). Food for thought: look back in anger – what clinical studies tell us about preclinical work. Altex 30: 275–291.

(4) Sutherland BA, 2012 Neuroprotection for ischaemic stroke: translation from the bench to the bedside. Int J Stroke 7: 407–18.

(5) Seok J, 2013 Genomic responses in mouse models poorly mimic human inflammatory diseases. PNAS 110: 3507–12.

(6) Arrowsmith J, 2012 A decade of change. Nat Rev Drug Discov 11:17–18

(7) Hay M, 2014 Clinical development success rates for investigational drugs. Nat Biotechnol 32 (1): 40–51.

On 2015 Aug 31, Wichor Bramer commented:

In my opinion the conclusions of this article are rather overdrawn. The authors have determined that the coverage of PubMed is high enough to be used in reviews. However, they have not performed searches for a systematic review, they have performed multiple searches for known items. If recall would be ideal PubMed would retrieve enough relevant articles to be used as a single database. However, recall is never ideal in any database. My recent observations (which have not yet been published) show that Medline retrieved 937 out of 1192 included references for 38 published reviews, so only 79%. Embase and medline together retrieved 1110 references (94%). So overall recall in embase is much better than in medline alone. In my opinion overall recall is not the best measure for database usefulness. When deciding a strategy for a systematic review, one decides for only one review, and not for 50 of 38. A better parameter is the minimum number observed. In medline the minimum was 53%, compared to 76% in embase/medline. Neither is acceptable in my opinion. The authors of this article also found that for some reviews, the coverage in pubmed was much lower than the total coverage. One database cannot be used to replace all other databases in the search for a systematic review.

On 2017 Feb 09, Kevin Hall commented:

The theoretical basis of the carbohydrate-insulin model (CIM) relies on generally accepted physiology about the endocrine regulation of adipose tissue based on short-term experiments lasting days and weeks. While there are indeed metabolic adaptations that take place on longer time scales, many of these changes actually support the conclusion that the purported metabolic advantages for body fat loss predicted by the CIM are inconsistent with the data.

For example, as evidence for a prolonged period of fat adaptation, Ludwig notes modest additional increases in blood and urine ketones observed after 1 week of either starvation Owen OE, 1983 or consuming a hypocaloric ketogenic diet Yang MU, 1976. The implication is that daily fat and ketone oxidation presumably increase along with their blood concentrations over extended time periods to eventually result in an acceleration of body fat loss with low carbohydrate high fat diets as predicted by the CIM. But since acceleration of fat loss during prolonged starvation would be counterproductive to survival, might there be data supporting a more physiological interpretation the prolonged increase in blood and urine ketones?

Both adipose lipolysis Bortz WM, 1972 and hepatic ketone production Balasse EO, 1989 reach a maximum within 1 week as demonstrated by isotopic tracer data. Therefore, rising blood ketone concentrations after 1 week must be explained by a reduced rate of removal from the blood. Indeed, muscle ketone oxidation decreases after 1 week of starvation and, along with decreased overall energy expenditure, the reduction in ketone oxidation results in rising blood concentrations and increased urinary excretion (page 144-152 of Burstztein S, et al. ‘Energy Metabolism, Indirect Calorimetry, and Nutrition.’ Williams & Wilkins 1989). Therefore, rather than being indicative of progressive mobilization of body fat to increase oxidation and accelerate fat loss, rising concentrations of blood ketones and fatty acids occurring after 1 week arise from reductions in ketone and fat oxidation concomitant with decreased energy expenditure.

The deleterious effects of a 600 kcal/d low carbohydrate ketogenic diet on body protein and lean mass were demonstrated in Vasquez JA, 1992 and were found to last about 1 month. Since weight loss was not significantly different compared to an isocaloric higher carbohydrate diet, body fat loss was likely attenuated during the ketogenic diet and therefore in direct opposition to the CIM predictions. Subsequent normalization of nitrogen balance would tend to result in an equivalent rate of body fat loss between the isocaloric diets over longer time periods. In Hall KD, 2016, urinary nitrogen excretion increased for 11 days after introducing a 2700 kcal/d ketogenic diet and coincided with attenuated body fat loss measured during the first 2 weeks of the diet. The rate of body fat loss appeared to normalize in the final 2 weeks, but did not exceed the fat loss observed during the isocaloric high carbohydrate run-in diet. Mere normalization of body fat and lean tissue loss over long time periods cannot compensate for early deficiencies. Therefore, these data run against CIM predictions of augmented fat loss with lower carbohydrate diets.

Ludwig uses linear extrapolation to claim that our data “would imply a 13 kg greater body fat loss versus the higher-fat diet over a year”. However, the same computational model that correctly predicted the difference in short-term body fat loss projected only small differences in long-term body fat between the diets. Based on these model simulations we concluded that “the body acts to minimize body fat differences with prolonged isocaloric diets varying in carbohydrate and fat.”

While I believe that outpatient weight loss trials demonstrate that low carbohydrate diets often outperform low fat diets over the short-term, there are little body weight differences over the long-term Freedhoff Y, 2016. However, outpatient studies cannot ensure or adequately measure diet adherence and therefore it is unclear whether greater short-term weight losses with low carbohydrate diets were due to reduced diet calories or the purported “metabolic advantages” of increased energy expenditure and augmented fat loss predicted by the CIM. The inpatient controlled feeding studies demonstrate that the observed short-term energy expenditure and body fat changes often violate CIM predictions.

Ludwig conveniently suggests that all existing inpatient controlled feeding studies have been too short and that longer duration studies might produce results more favorable to the CIM. But even this were true, the current data demonstrate repeated violations of CIM model predictions and constitute experimental falsifications of the CIM. This possibility was accurately described in my review Hall KD, 2017 and requires an ad hoc modification of the CIM such that the metabolic advantages of isocaloric lower carbohydrate diets only begin after a time lag lasting many weeks – a possibility currently unsupported by data but obviously supported by sincere belief.

On 2017 Feb 07, DAVID LUDWIG commented:

In his comment of 31 January 2017, Hall continues to insist that the results of his 6-day study and other very short feeding studies of substrate oxidation inform understanding of the long-term relationship between diet and body composition. This contention can be simply dismissed, with recognition that the 36 g/d advantage in fat oxidation on Hall’s low-fat diet would imply a 13 kg greater body fat loss versus the higher-fat diet over a year. There is simply no precedent for such an effect, and if anything the long-term clinical trials suggest the opposite Tobias DK, 2015 Mansoor N, 2016 Mancini JG, 2016 Sackner-Bernstein J, 2015 Bueno NB, 2013.

The reason short term studies of high-fat diets are misleading is that the process of adapting to reduced carbohydrate intake can take several weeks. We can clearly observe this phenomenon in 4 published graphs involving very-low-carbohydrate diets.

For convenience, these figures can be viewed at this link:

Owen OE, 1983 Figure 1. Ketones are, of course, the hallmark of adaptation to a very-low-carbohydrate (ketogenic) diet. Generally speaking, the most potent stimulus of ketosis is fasting, since the consumption of all gluconeogenic precursors (carbohydrate and protein) is zero. As this figure shows, the blood levels of each of the three ketone species (BOHB, AcAc and acetone) continues to rise for ≥3 weeks. Indeed, the prolonged nature of adaptation to complete fasting has been known since the classic starvation studies of Cahill GF Jr, 1971. It stands to reason that this process might take even longer on standard low-carbohydrate diets, which inevitably provide ≥ 20 g carbohydrate/d and substantial protein.

Yang MU, 1976 Figure 3A. Among men with obesity on an 800 kcal/d ketogenic diet (10 g/d carbohydrate, 50 g/d protein), urinary ketones continued to rise for 10 days through the end of the experiment, and by that point had achieved levels equivalent only to those on day 4 of complete fasting. Presumably, this process would be even slower with a non-calorie restricted ketogenic diet (because of inevitably higher carbohydrate and protein content).

Vazquez JA, 1992 Figure 5B. On a conventional high-carbohydrate diet, the brain is critically dependent on glucose. With acute restriction of dietary carbohydrate (by fasting or a ketogenic diet), the body obtains gluconeogenic precursors by breaking down muscle. However, with rising ketone concentrations, the brain becomes adapted, sparing glucose. In this way, the body shifts away from protein to fat metabolism, sparing lean tissue. This process is clearly depicted among women with obesity given a calorie-restricted ketogenic diet (10 g carbohydrate/d) vs a nonketogenic diet (76 g carbohydrate/d), both with protein 50 g protein/d. For 3 weeks, nitrogen balance was strongly negative on the ketogenic diet compared to the non-ketogenic diet, but this difference was completely abolished by week 4. What would subsequently happen? We simply can’t know from the short-term studies.

Hall KD, 2016 Figure 2B. Another study by Hall shows that the transient decrease in rate of fat loss upon initiation of the ketogenic diet accelerates after 2 weeks.

The existence of this prolonged adaptive process explains why metabolic advantages for low-fat diet are consistently seen in very short metabolic studies. But after 2 to 4 weeks, advantages for low-carbohydrate diets begin to emerge, Hall KD, 2016 Miyashita Y, 2004 Ebbeling CB, 2012. Any meaningful conclusions about the long-term effects of macronutrients must await longer studies.

On 2017 Jan 31, Kevin Hall commented:

My recent review of the carbohydrate-insulin model Hall KD, 2017 presented a synthesis of the evidence from 20 inpatient controlled feeding studies strongly suggesting that at least some important aspects of the model are in need of modification. In particular, our recent studies Hall KD, 2015, Hall KD, 2016 employing carefully controlled inpatient isocaloric diets with constant protein, but differing in carbohydrate and fat, resulted in statistically significant differences between the diets regarding body fat and energy expenditure that were in directions opposite to predictions of the carbohydrate-insulin model.

Ludwig comments that the diets used in Hall KD, 2015 were either too low in fat or insufficiently low in carbohydrate. However, while these considerations may be clinically important for sustainability of the diets, they are irrelevant to whether the diets resulted in a valid test of the carbohydrate-insulin model predictions. We selectively reduced 30% of baseline calories solely by restricting either carbohydrate or fat. These diets achieved substantial differences in daily insulin secretion as measured by ~20% lower 24hr urinary C-peptide excretion with the reduced carbohydrate diet as compared with the reduced fat diet (p= 0.001) which was unchanged from baseline. Whereas the reduced fat diet resulted in no significant energy expenditure changes from baseline, carbohydrate restriction resulted in a ~100 kcal/d decrease in both daily energy expenditure and sleeping metabolic rate. These results were in direct opposition to the carbohydrate-insulin model predictions, but in accord with the previous studies described in the review as well as a subsequent study demonstrating that lower insulin secretion was associated with a greater reduction of metabolic rate during weight loss Muller MJ, 2015.

While the DXA methodology was not sufficiently precise to detect significant differences in body fat loss between the diets, even this null result runs counter to the predicted greater body fat loss with the reduced carbohydrate diet. Importantly, the highly sensitive fat balance technique demonstrated small but statistically significant differences in cumulative body fat loss (p<0.0001) in the direction opposite to the carbohydrate-insulin model predictions. Ludwig claims that our results are invalid because “rates of fat oxidation, the primary endpoint, are exquisitely sensitive to energy balance. A miscalculation of available energy for each diet of 5% in opposite directions could explain the study’s findings.” However, it is highly implausible that small uncertainties in the metabolizable energy content of the diet amounting to <100 kcal/d could explain the >400 kcal/d (p<0.0001) measured difference in daily fat oxidation rate. Furthermore, our results were robust to the study errors and exclusions described in the report and our observations clearly falsified important aspects of the carbohydrate-insulin model.

Ludwig argues that “it can take the body weeks to fully adapt to a high fat diet”, However, daily fat oxidation has been observed to plateau within the first week when added dietary fat is accompanied by an isocaloric reduction in carbohydrate as indicated by the rapid and sustained drop in daily respiratory quotient in Hall KD, 2016 and Schrauwen P, 1997. Similarly, Hall KD, 2015 observed a decrease and plateau in daily respiratory quotient with the reduced carbohydrate diet, whereas the reduced fat diet resulted in no significant changes indicating that daily fat oxidation was unaffected. As further evidence that adaptations to carbohydrate restriction occur relatively quickly, adipose tissue lipolysis is known to reach a maximum within the first week of a prolonged fast Bortz WM, 1972 as does hepatic ketone production Balasse EO, 1989.

While there is no evidence that carbohydrate restricted diets lead to an acceleration of daily fat oxidation on time scales longer than 1 week, and there is no known physiological mechanism for such an effect, this possibility cannot be ruled out. Such speculative long term effects constitute an ad hoc modification of the carbohydrate-insulin model whereby violations of model predictions on time scales of 1 month or less are somehow reversed.

Ludwig is correct that it takes the body a long time to equilibrate to added dietary fat because, unlike carbohydrate and protein, dietary fat does not directly promote its own oxidation and does not significantly increase daily energy expenditure Schutz Y, 1989 and Horton TJ, 1995. Unfortunately, these observations run counter to carbohydrate-insulin model predictions because they imply that added dietary fat results in a particularly efficient means to accumulate body fat compared to added carbohydrate or protein Bray GA, 2012. If such an added fat diet is sustained, adipose tissue will continue to expand until lipolysis is increased to sufficiently elevate circulating fatty acids and thereby increase daily fat oxidation to reestablish balance with fat intake Flatt JP, 1988.

Of course, differences in long term ad libitum food intake between diets varying in macronutrient composition could either obviate or amplify any predicted body fat differences based solely on fat oxidation or energy expenditure considerations. Such mechanisms warrant further investigation and will inform improved models of obesity. Nevertheless, it is clear that several important aspects of the carbohydrate-insulin model have been experimentally falsified by a variety of studies, including Hall KD, 2015.

On 2017 Jan 17, DAVID LUDWIG commented:

In a recent review, the first author of this Cell Metabolism article (Kevin Hall) cites this 6-day study as a basis for having “falsified” the Carbohydrate-Insulin Model of obesity. That argument disregards some key limitations of this study, which warrant elucidation.

In the discussion section of this study, Hall and colleagues write: “Our relatively short-term experimental study has obvious limitations in its ability to translate to fat mass changes over prolonged durations” (NB, it can take the body weeks to fully adapt to a high fat diet Hawley JA, 2011 Vazquez JA, 1992 Veum VL, 2017). Beyond short duration and confounding by transient biological adaptations, the study: 1) did not find a difference in actual fat mass by DXA (p=0.78); 2) used an exceptionally low fat content for the low-fat diet (< 8% of total energy), arguably without precedent in any population consuming natural diets; 3) used a relatively mild restriction of carbohydrate (30% of total energy), well short of typical very-low-carbohydrate diets; 4) had protocol errors and post-randomization data exclusions that could confound findings; and 5) failed to verify biologically available energy of the diet (e.g., by analysis of the diets and stools for energy content). Regarding this last point, rates of fat oxidation, the primary endpoint, are exquisitely sensitive to energy balance. A miscalculation of available energy for each diet of 5% in opposite directions could explain the study’s findings – and this possibility can’t be ruled out in studies of such short duration.

Thus, this study should not be interpreted as providing a definitive test of the Carbohydrate-Insulin Model.

On 2016 Aug 23, Ben Goldacre commented:

One of the trials in this article has the wrong trial registry ID associated with it on PubMed: both in the XML on PubMed, and in the originating journal article. The ID given is NCT0050446. We believe the correct ID, which we have found by hand searching, is NCT00550446.

This comment is being posted as part of the OpenTrials.net project^[1] , an open database threading together all publicly accessible documents and data on each trial, globally. In the course of creating the database, and matching documents and data sources about trials from different locations, we have identified various anomalies in datasets such as PubMed, and in published papers. Alongside documenting the prevalence of problems, we are also attempting to correct these errors and anomalies wherever possible, by feeding back to the originators. We have corrected this data in the OpenTrials.net database; we hope that this trial’s text and metadata can also be corrected at source, in PubMed and in the accompanying paper.

Many thanks,

Jessica Fleminger, Ben Goldacre*

[1] Goldacre, B., Gray, J., 2016. OpenTrials: towards a collaborative open database of all available information on all clinical trials. Trials 17. doi:10.1186/s13063-016-1290-8 PMID: 27056367

* Dr Ben Goldacre BA MA MSc MBBS MRCPsych<br> Senior Clinical Research Fellow<br> ben.goldacre@phc.ox.ac.uk<br> www.ebmDataLab.net<br> Centre for Evidence Based Medicine<br> Department of Primary Care Health Sciences<br> University of Oxford<br> Radcliffe Observatory Quarter<br> Woodstock Road<br> Oxford OX2 6GG

On 2016 Jun 13, Stuart Phillips commented:

See a letter to the editor "Greater electromyographic responses do not imply greater motor unit recruitment and 'hypertrophic potential' cannot be inferred" regarding the data in this paper: http://www.ncbi.nlm.nih.gov/pubmed/26670996

On 2015 Oct 07, Siddharudha Shivalli commented:

Importance of study tool validation and adherence to reporting guidelines in community based cross sectional studies

I read the article titled “Prevalence and risk factors associated with malaria infection among pregnant women in a semi-urban community of north-western Nigeria” by Fana SA et al, with curiosity. Authors’ efforts are admirable. This study reiterates malaria as a major public health problem among pregnant women in Argungu and lack of education and non-usage of ITNs augments the risk of malaria. However, following issues need to be addressed. In methods section, authors mention that 266 pregnant women in their second trimester were randomly selected from the enlisted 850 households. But, following should have been mentioned : How many pregnant women were assessed for eligibility? Why only 2nd trimester pregnant women were included? What was the operational definition to categorize a pregnant woman as user or non-user of ITN? Prevalence of malaria, a key outcome variable, should have been reported with 95% confidence intervals . In results section, authors have repeatedly mentioned the p value as ‘0.000’. SPSS, by default setting, displays p value as zero if it extends beyond 3 decimal points (i.e. p=0.0000007 would be displayed as p=0.00). Practically, the value of p cannot be zero and hence, I, would suggest to report it as p<0.0001. Authors have mentioned in the limitation as they did not assess the key factors such as gravidity, trimester, whether IPT was given or not and frequency of antenatal care visits etc. While conducting a community based study with a sample frame of 850 households, one must be sure of the sampling and the study tool/s. The questionnaire should have been validated by 3/more epidemiologists. Use of the phrase ‘risk factors’ in the title is debatable as it was a cross sectional study and the observed associations may not imply causality.<br> None the less, I must congratulate the authors for investigating an important public health problem among pregnant women.

Conflict of Interests: The author declares that there is no conflict of interest about this publication.

On 2015 Sep 08, Kausik Datta commented:

The authors make it a point to state: "Our results highlight (i) the scholarship and rational methodology of premodern medical professionals and (ii) the untapped potential of premodern remedies for yielding novel therapeutics at a time when new antibiotics are desperately needed."

Can someone kindly explain to me how this study, while immensely interesting to me, is any different from regular ethnobotany or pharmacognosy studies?

On 2015 Aug 24, Johannes Weinberger commented:

Accession number for all sequencing data of this study: NCBI Sequence Read Archive (SRA): PRJNA283508

On 2015 Oct 01, Jesus Castagnetto commented:

One minor correction. This article mentions a program developed in Peru, but calls it "AMUATA" instead of the correct name "AMAUTA".

On 2015 Aug 27, Anthony Michael commented:

The authors of this paper claim that this is the first report of heterologous production of 1,3-diaminopropane in E. coli. They appear to have overlooked our report of heterologous production of 1,3-diaminopropane in E. coli published six years ago in the Journal of Biological Chemistry: http://www.ncbi.nlm.nih.gov/pubmed/19196710 This is an understandable oversight considering the crowded field of heterologous production of 1,3-diaminopropane in E. coli research.

On 2015 Nov 12, RUBEN ABAGYAN commented:

Additional disclosure. One of the authors, R.A., has an equity interest in Molsoft, LLC. The terms of this arrangement have been reviewed and approved by the University of California, San Diego in accordance with its conflict of interest policies.

On 2015 Oct 21, Michael McCann commented:

I would like to point out an earlier paper, Entezari A, 2012, which also develops theory for using splines to discretize tomography reconstruction. I'm wondering if the authors can briefly comment about the relationship between the two approaches (which I think boils down to the way projection is handled).

(Disclosure: though I'm not an author of Entezari A, 2012, I am aware of the paper because I'm currently working in that group.)

On 2016 Mar 31, Lydia Maniatis commented:

Todd, Egan & Kallie (2015) provide more demonstrations of violations of the "darker-is-deeper" rule. They note that the rule is already falsified by literature: "the evidence is overwhelming that observers' judgments of 3D shape from shading do not conform to the predictions of [the rule]" (Of course - the possible falsifying cases are infinite in number).

They also note that the rule was falsified by Langer and Bulthoff (2000), who concluded from their observations that "the perception of shape from shading must be based on a process that is more sophisticated that a simple darker-is-deeper heuristic." (Todd et al, 2015).

I wondered whether Chen and Tyler (2015) had cited Langer & Bulthoff (2000) in this paper. Indeed, they do, but the implication is the opposite of the description cited above:

"Langer & Bülthoff (2000) showed that the observer can indeed discriminate between “hills” and “valleys” on a surface with this “dark-is-deep” rule" (Chen and Tyler, 2015). This description conveys a very different and misleading (I would prefer to describe it as dishonest) impression of the results of the cited reference, papering over complications and contradictions to smooth the way to a preferred but non-viable argument.

On 2015 Nov 27, Lydia Maniatis commented:

I just want to expand a little on my previous comment, with regard to the direction of the light source. As I said, the implied direction of the light is front-parallel to the corrugations, but my suggestion that the shadowing is consistent with attenuation due to distance along the normal seems inadequate. I want to add that the putative peaks of the corrugations would tend to reflect light from all directions, not only light coming along the normal, and that the troughs would receive less light because, from some directions, it would be blocked by the protruding sections. (I realise now that this is the point the author was trying to make with the north-south wall example. He is correct but again, that does not mean that the light is not directional, only that it comes from a number of directions. I think that my assumption of fronto-parallel lighting may be more consistent with the patterns than his assumption that the directions form the surface of a half-sphere (the sky), but they might be equivalent.)

On 2015 Nov 27, Lydia Maniatis commented:

In his comment above, CCC says that “The issue of diffuse illumination is a rather minor aspect of the paper...” However, based on the abstract, confirming the “diffuse illumination assumption" is presented as central goal: “These results validate the idea that human observers can use the diffuse illumination assumption to perceived depth from luminance gradients alone without making an assumption of light direction.”

In the authors' stimuli, light does, in fact, have an implied direction – the light source is fronto-parallel to the (apparent) corrugations, such that its intensity is attenuated with distance in the direction along the normal.

As a general fact, the greater weighting of what we could call pictorial cues over binocular disparity in the perception of depth did not require more corroboration. It is evident in the fact that very convincing pictorial depth effects are easy to achieve, despite the zero disparity. Also, with regard to the presence of positive (rather than zero) disparity cues, a relevant reference that was not included is Pizlo, Zygmunt, Yunfeng Li, and Robert M. Steinman. "Binocular disparity only comes into play when everything else fails; a finding with broader implications than one might suppose." Spatial Vision 21.6 (2008): 495-508. (Update 12/2: I apologise, the authors do cite this paper).

In their abstract Langer and Buelthoff (2000) conclude that “overall performance in the depth-discrimination task was superior to that predicted by a dark-means-deep model. This implies that humans use a more accurate model than dark-means-deep to perceive shape-from-shading under diffuse lighting.”

On 2015 Nov 23, Chien-Chung Chen commented:

First of all, it should be noted that this paper is about cue combination, not shape-from-shading per se. The issue of diffuse illumination is a rather minor aspect of the paper, to which most of the discussion in the thread is quite irrelevant.

The commenter’s argument that no shadows are generated under diffuse illumination is a serious misunderstanding of the properties of diffuse illumination. Although, the light comes from every direction, it does not reach everywhere. For instance, suppose one erects a wall running from north to south on an otherwise flat surface. Immediately to the west of the wall, all the light from the east would be blocked by the wall. Thus, this location would receive less light and in turn appear darker than it would be without the wall, effectively forming a diffuse shadow. In short, the light received at a position on a surface depends on the extent of the sky it is exposed to. On an uneven surface, a position in a “valley” would “see” less of the sky than a position on a “hill” and thus would appear darker. This is why under diffuse illumination, darker implies deeper. Notice that, deeper means deeper than the surrounding surface, not from the source of light as the commenter erroneously states.

The difference between the “light-from-above” but with “darker-is-deeper” assumptions is one of the range of scenes to which they apply. It is indeed an empirical issue whether an interpretation that applies to many scenes is a true default or is cued by some aspect of the scene. Our claim is that the directional lighting assumption that is so common in computer graphics is typically cued by the discrepancy between symmetric contour information and asymmetric shading information. (In the case of Ramachandran’s disks, the contour information is circularly symmetric.) This particular discrepancy is a narrow subset of all possible arrangements of contours and shading in images. If this discrepancy is removed, either by making the shading (circularly) symmetric or by making the contours asymmetric, the prediction is that the visual system will not maintain the “light-from-above” assumption but will default to the “darker-is-deeper” assumption. The “darker-is-deeper” assumption is considered the more basic default because it can apply to images with any degree of symmetry or asymmetry. Of all possible visual scenes generated at random, only a small subset will contain symmetries and an even smaller subset will have contour/shading discrepancies consistent with any kind of oblique lighting direction. It is only in the presence of such discrepancies that a default for the lighting direction could come into play.

Finally, it is true that is difficult to separate “darker-is-deeper” and “darker-is-darker-colored” from one instance. However, such heuristics do not depend on one instance but the statistics of nature scenes. In a complex scene, one does not just use one visual cue. “Good shape” is not a necessary condition for darker-is-deeper rule as the complex random stimuli used by Langer & Buelthoff (2000) showed.

On 2015 Sep 26, Lydia Maniatis commented:

You say in the article that under diffuse illumination "light comes from every direction." In that case, there should be no shadows; a uniformly-colored surface should appear uniformly-colored in the image and unevenness (bumps, etc) will not be evident. The fact that you postulate a “darker-deeper” rule indicates that illumination is directional – “deeper” being equivalent to “farther from the source.”

When you say that diffuse illumination is “the default assumption when there is not strong enough evidence to support other interpretations of the scene,” how is this different from my saying that “light from the top right” is the default assumption when there is not strong enough evidence to support other interpretations of the scene?

The information available to the visual system is simply a luminance pattern, which could derive from an infinite number of reflectance/illumination combinations at each point. A “diffuse illumination” assumption – or any illumination assumption – cannot drive a solution to this problem, because it actually cannot decide between the possibilities: “darker is deeper” and “darker-is-darker-colored.” The drivers of the solution are good shape assumptions, combined with the assumption that good shapes have uniform reflectance. So if we have a good shape, (e.g. the Ramachandran bumps) then we assume that the luminance differences are illumination-based, and the illumination assumption follows.

(I would like to add that my earlier comment was deleted by moderators b/c it was a duplication, not b/c it was inappropriate!)

On 2015 Sep 25, Chien-Chung Chen commented:

The commenter seemed to confuse the concept of “default” with “predominate”. The default assumption is like a null hypothesis in statistics. The visual system uses the default assumption when there is not strong enough evidence to support other interpretations of the scene. The commenter seemed to take it that the default assumption meant that the observer would always make this assumption about the scene over other possible interpretations (thus, her statement that we “could not explain why we often perceive illumination to be directional”). With this clarification, it should be clear that in our paper, there is no logical contradiction in our claim that diffused illumination is a visual system default.

On 2015 Aug 18, Lydia Maniatis commented:

The authors claim to their results a. "validate the idea that human observers can use the diffuse illumination assumption to perceive depth from luminance gradients alone without making an assumption of light direction and b. confirm that "observers can recover shape from shading under diffuse illumination on the basis of the “dark-is-deep” rule."

There is a logical problem with the idea that the visual system has a "diffuse illumination assumption," on the basis of which it applies a dark is deep rule.

In viewing a scene, we perceive both the quality of the illumination (including direction) and the albedo of surfaces. Both of these qualities are constructed by the visual system, which must resolve the reflectance/illumination ambiguity. There is no default assumption as to either the reflectance of a particular patch nor the quality of the illumination. Both are inferred on the basis of structural constraints.

For example, in the case of the Ramachandran bumps demonstration, the fundamental choice is between seeing concave or convex circles with a homogeneous reflectance, or seeing part moon shapes of varying reflectances (some dark, some light, some in-between). In the Chen/Tyler stimuli, structural assumptions as to the "best" shape consistent with the stimulus produce a percept consistent with a homogeneously coloured surface under diffuse illumination. If the diffuse illumination assumption came first, we could not explain why we see the corrugations in the obviously flat picture, and we could not explain why we often perceive illumination to be directional. In addition, we can create cartoonish versions of corrugations, in which the deeper dark region is not signalled by a gradient, and still produce a similar impression of dark is deep. If structural assumptions don't favour it, however, dark will not be deep, it will just be dark.

In order to defend an a priori diffuse illumination assumption, the authors would need to explain why the assumption is made only sometimes (i.e. only when structural constraints demand it. )

I would add that their stimuli are not really consistent with diffuse illumination conditions, in the sense that both the upper and lower edges are straight, and thus we must be seeing the corrugated surfaces through a rectangular aperture. The concave sections of the surface would be at some distance from the edge of this aperture and I assume the luminance profile would indicate this in some way, with shadowing, or black holes in the gaps, something.

When it comes to 3D perception, shading follows shape, not vice versa.

On 2015 Aug 19, Seyed Moayed Alavian commented:

Dear Author, I would like to emphasize the role of supervision on barbers and more efforts for increasing the people awareness . Yours Prof Alavian

On 2015 Dec 09, Donald Forsdyke commented:

PURINE LOADING AS A THERMAL ADAPTATION The proteins of thermophiles are generally more heat-stable than the corresponding proteins from mesophiles. This must be reflected in either, or both, of two major amino acid variables – composition and order. In the past the notion that amino acid composition might be reflective of the pressure in thermophiles to retain purine-rich codons (3) has been disparaged by Zeldovich et al. (6). In this elegant new paper (5), Venev and Zeldovich (2015) agree that the “multiple factors” not accounted for in their modelling “include the influence of the genetic code and guanine-cytosine (GC) content of the genomes on amino acid frequencies.” However, there is puzzlement that the “theory and simulations predict a strong increase of leucine content in the thermostable proteins, whereas it is only minimally increased in experimental data.” Perhaps it is of relevance that leucine is on the top left quadrant of the standard presentation of the genic code, its codons being extremely poor in purines.

My response to Zeldovich et al. (6) in 2007, and my follow-up references in 2012 (1, 4), are set out below. One of the coauthors of the 2007 paper has recently further contributed to this topic (2).

2007 Response

This paper draws conclusions tending to oppose those of myself and coworkers (cited). A "key question" is held to be: "Which factor - amino acid or nucleotide composition - is primary in thermal adaptation and which is derivative?" Previous evidence is considered "anecdotal." Now there is evidence for "an exact and conclusive" relationship, based on an "exhaustive study" that provides a "complete picture." A set of amino acids - IVYWREL - correlates well with growth temperature. It is noted:

"Signatures of thermal adaptation in protein sequences can be due to the specific biases in nucleotide sequences and vice versa. ... One has to explore whether a specific composition of nucleotide (amino acid) sequences shapes the content of amino acid (nucleotide) ones, or thermal adaptation of proteins and DNA (at the level of sequence compositions) are independent processes."

In other words, are primary adaptations at the nucleic acid level driving changes at the protein level, or vice- versa? To what extent are the two processes independent? Their conclusion:

"Resolving the old-standing controversy, we determined that the variation in nucleotide composition (increase of purine-load, or A + G content with temperature) is largely a consequence of thermal adaptation of proteins."

Thus, the superficial reader of the paper, while noting the purine-richness of some of the codons corresponding to the IVYWREL amino acids, will conclude that the "independent processes" alternative has been excluded. Reading the paper (e.g. Figure 7) one can question the validity of this conclusion. Many of the IVYWREL amino acids have purine-poor alternative codons (especially IYLV, which at best can only change one purine unit in their codons). One of the IVYWREL amino acids has relatively purine-rich alternative codons (R, which at best can change two purine units). Two (EW) are always purine-rich, and there are no alternatives.

Displaying more EW's as the temperature got hotter would satisfy a need both for more purines and for more tryptophan and glutamate, so here there is no discrimination as to whether one "shapes" the organism’s content of the other. Displaying more IYLVs gives only minimal flexibility in accommodating a purine-need. Most flexibility is provided by R codons.

The authors do not give statistics for the differences between the slopes of Figs. 7a (unshuffled codons) and 7b (shuffled codons), but they appear real, presumably reflecting the choice biologically of purine-rich codons, a choice the organisms might not have to make if there were no independent purine-loading pressure. Thus, the authors note, but only in parenthesis, that the slopes "are somewhat different suggesting that codon bias may be partly responsible for the overall purine composition of DNA."

2012 Response

As a follow up, it can be noted that Dehouck et al. (2008) report that relationship between a protein's thermostability and the optimum growth temperature of the organism containing it, is not so close as previously thought (1). Furthermore, Liu et al. (2012) now conclude from a study of xylanase purine-rich coding sequences that "The codons relating to enzyme thermal property are selected by thermophilic force at [the] nucleotide level," not at the protein level (4).

1.Dehouck Y, Folch B, Rooman M (2008) Revisiting the correlation between proteins' thermoresistance and organisms' thermophilicity. Protein Engineering, Design and Selection 21:275-278.Dehouck Y, 2008

2.Goncearenco A, Berezofsky IN (2014) The fundamental tradeoff in genomes and proteomes of prokaryotes established by the genetic code, codon entropy, and the physics of nucleic acids and proteins. Biology Direct 9:29 Goncearenco A, 2014

3.Lambros RJ, Mortimer JR, Forsdyke DR (2003) Optimum growth temperature and the base composition of open reading frames in prokaryotes. Extremophiles 7:443–450.Lambros RJ, 2003

4.Liu L, Wang L, Zhang Z, Wang S, Chen H (2012) Effect of codon message on xylanase thermal activity. J. Biol. Chem. 287:27183-27188 Liu L, 2012

5.Venev SV, Zeldovich KB (2015) Massive parallel sampling of lattice proteins reveals foundations of thermal adaptation. J. Chem. Phys. 143: 055101Venev SV, 2015

6.Zeldovich KB, Berezofsky IN, Shakhnovich EI (2007) Protein and DNA sequence determinants of thermophilic adaptation. PLOS Comput. Biol. 3(1), e5.Zeldovich KB, 2007

On 2015 Sep 23, Klaas Vandepoele commented:

Datasets described in the paper are available as BED files via http://bioinformatics.intec.ugent.be/blsspeller/ :

BLS files (contain phylogenetic conserved genomic regions given certain cutoffs, in bed file format): osa/zma_C90F20B15.bed

Conserved known motif files (conserved known motif files, from CisBP database v1.02): osa/zmaconservedknownmotifsBLS15/95.bed

DNase hypersensitive sites (regions of open chromatin, in bed file format): DNase1hypersensitivesites_osa.bed

On 2015 Aug 09, Andrzej Miskiewicz commented:

The clinical study fall in line with the previously published genetic research of Zhang L. and Wong D.T. However in the presented manuscript the authors focus their research on the inflammation mechanism as a possible molecular mechanism common for the mentioned conditions.

On 2015 Aug 12, Donald Forsdyke commented:

WINGE PROPOSED HYBRID STERILITY CURED BY WHOLE-GENOME DUPLICATION

That hybrid sterility would be 'cured' by whole genome duplication was suggested by 'the father of yeast genetics' [1], Őjvind Winge [2], and has been extensively discussed in modern texts on speciation [3] and evolutionary biology [4].

He would doubtless have been delighted that his favorite organism (apart from dogs) had formed the basis of the elegant study by Marcet-Houben and Gabaldon that provides a welcome endorsement of his viewpoint.

[1] Szybalski W (2001) My road to Őjvind Winge, the father of yeast genetics. Genetics 158:1–6.

[2] Winge Ő (1917) The chromosomes. Their numbers and general importance. Comptes Rendus des Travaux du Laboratoire Carlsberg. 13:131–275. see Webpage.

[3] Forsdyke DR (2001) The Origin of Species Revisited. Montreal: McGill-Queen’s University Press, pp. 72–79.

[4] Forsdyke DR (2011) Evolutionary Bioinformatics. 2nd edition. New York: Springer, pp. 184–186.

On 2015 Aug 27, Trevor Marshall commented:

I see that Figure 2 of this paper closely resembles the in-silico image of D-Binding Protein which I presented (inter alia) at the Autoimmunity Congress in 2008, in Porto. I wonder if it was attributed properly? The caption I see on the journal's website does not seem to give any citation at all. Additionally, the text seems to be describing my image as the VDR, whereas this image is actually of the D-Binding Protein, not the VDR, despite the caption: "Figura 2 di 2. Il recettore della vitamina D (VDR) [D Binding Protein Complexed with 1,25-D]"

On 2017 Jul 10, Randi Pechacek commented:

Elisabeth Bik discussed this paper in a microBEnet blog about microbial biofilms in water meters.

On 2015 Nov 24, Salzman Lab Journal Club commented:

This thought-provoking paper describes a potential mechanism for cryptic exon splicing due to TDP-43 proteinopathy. It is interesting to consider the evolutionary history of the sequence surrounding these cryptic exons and the origins of TDP-43 binding. The authors show that GU-repeats, the consensus motif for TDP-43, often flank these cryptically spliced exons. Is it that these GU repeats originally promoted the splicing of these cryptic exons and TDP-43 evolved the ability to bind GU repeats to restrict this process or did the GU-repeats arise later to repress the splicing by recruiting TDP-43, leading to repression? Additionally, we would be interested in seeing the effect of other engineered TDP-43 mutants and truncations on cryptically spliced exons, specifically, a mutant lacking the C-terminus and other splicing repressor domains and the full length TDP-43 rescue as simple controls.

On 2015 Sep 25, David Keller commented:

More observational evidence suggesting cardiovascular benefits associated with testosterone replacement

This observational study found that TRT (testosterone replacement therapy) dosing resulting in normalization of serum testosterone levels was associated with greater benefits than were lower doses. Association of an intervention with a benefit in an observational study does not prove causation and cannot be used to prove the safety or efficacy of the intervention. Observational studies are inherently affected by confounding variables which can only be eliminated by conducting a randomized controlled trial of the intervention.

Indeed, the authors conclude that "adequately powered, prospective, well-designed trials with a long-term follow-up will be needed to reach a conclusive agreement regarding the effect of TRT on CV risk."

We certainly have more than enough results from observational studies at this time to justify the expense of a large prospective randomized trial of testosterone replacement therapy. Given the possibility that the cardiovascular benefits associated with TRT may be caused by TRT, it is a major disservice to ageing men to delay the definitive randomized trial any longer.

On 2016 Jan 25, Noa Krawczyk commented:

Versão em Português do artigo disponível [Portuguese version available at]: https://figshare.com/s/f82b748e5f4133e4af2c

"A interação entre comportamentos de consumo de droga, contextos e acesso aos cuidados de saúde: Um estudo qualitativo explorando atitudes e experiências de usuários de crack no Rio de Janeiro e São Paulo, Brasil"

Resumo:

Antecedentes: Apesar da crescente atenção em torno do uso de crack no Brasil, pouco se sabe sobre as histórias dos usuários, seus padrões de consumo, e a interação de hábitos de consumo de droga, contextos e acesso/barreiras ao(s) cuidados de saúde. Estudos qualitativos raramente comparam os achados de pessoas que usam crack a partir de diferentes contextos. Este estudo tem como objetivo explorar os insights de usuários regulares de crack em duas grandes cidades brasileiras e examinar como fatores sociais e contextuais, incluindo o estigma e a marginalização, influenciam o uso inicial e diversos problemas de saúde e sociais.

Métodos: Entrevistas em profundidade e grupos focais foram realizados com 38 adultos usuários de crack recrutados em bairros pobres do Rio de Janeiro e São Paulo. As entrevistas e grupos focais foram gravadas em áudio e transcritas na íntegra. Procedeuse à análise qualitativa e os conteúdos foram organizados e analisados por temas recorrentes relevantes para os interesses do estudo.

Resultados: Para os participantes do estudo de ambas as cidades, o uso frequente de crack desempenha um papel central na vida diária e leva a uma série de consequências físicas, psicológicas e sociais. Os interesses comuns entre os usuários incluem o uso excessivo de crack, o engajamento em comportamentos de risco, a utilização pouco frequente de serviços de saúde, a marginalização e a dificuldade em reduzir o consumo de drogas.

Conclusões: As condições desfavoráveis em que muitos usuários de crack crescem e vivem podem perpetuar os comportamentos de risco, e o estigma marginaliza ainda mais os usuários dos serviços de saúde e de recuperação necessários. A redução do estigma e do discurso moralizante relacionado ao uso de drogas, especialmente entre os profissionais de saúde e policiais, pode incentivar os usuários a procurar atendimento necessário. Novas alternativas de cuidado para usuários marginalizados, baseados em redução de danos estão sendo desenvolvidas em algumas localidades no Brasil e outros países, e deveriam ser adaptadas e expandidas para populações vulneráveis.

On 2015 Sep 05, Andrew R Kniss commented:

It is certainly plausible that herbicides (glyphosate or other) might have some direct effect on earthworms. However, due to the design flaws, these effects cannot be evaluated from this particular study. The problems with this paper boil down to two main points:

1) One of the herbicides applied in the study was "Roundup Speed" which contains the herbicide pelargonic acid in addition to glyphosate, so it is impossible to conclude anything about the direct effects of glyphosate. 2) More importantly, the researchers didn't include a control treatment where they killed the plants without herbicides. All of the effects on earthworms and nutrients observed in this study could simply be due to killing the plants. It is perfectly plausible the exact same effects would be observed if the plants were clipped or pulled out of the pots.

In addition, the glyphosate rate used in this study is far greater than would be used in field applications of this product. I calculated the amount of glyphosate applied to the pots (adding up the three applications they made) and converted it into the amount of glyphosate per unit area. It turns out the amount of glyphosate applied to each pot is equivalent to a field rate of 12,680 grams per hectare. A typical application rate in a field of glyphosate-resistant crops would be somewhere between 800 to 1,300 grams per hectare. So the amount of glyphosate they applied is about an order of magnitude too high to be relevant to most field situations.

Blog post with more detail: http://weedcontrolfreaks.com/2015/09/dead-plants-are-probably-bad-for-earthworms/

On 2015 Oct 16, Andrea Messori commented:

How to manage the price of the newest expensive agents approved for HCV therapy? Pharmaceutical firms do not adopt the same policies across different countries and different regions

by Andrea Messori, PharmD, Sabrina Trippoli, PharmD, Claudio Marinai, Pharm D

HTA Unit, Tuscany Region, ESTAR, Regional Health Service, 50100 Firenze, Italy

In managing the price of the newest expensive agents approved for HCV therapy, Scripts (the largest pharmacy benefit manager in the US) was successful in fostering the competition between Abbvie and Gilead. In particular, Abbvie accepted an exchange of more prescriptions for price rebates, and so Scripts decided to cover Viekira Pak for the majority of HCV patients and restricted the coverage of Gilead products only under certain exceptions (1).

While Wilensky (1) emphasizes that this strategy based on competition in prices can be an effective way of reducing the cost of these expensive treatments, other experiences in this field have not been successful and therefore deserve to be mentioned. In May 2015, the Tuscany region of Italy undertook a competitive tender scheme aimed at the prescriptions for 18,000 HCV patients of our region (those without cirrhosis) (2) in which both Abbvie and Gilead were expected to participate. In fact, at national level the majority of reimbursed treatments for HCV (currently restricted essentially to patients with cirrhosis) are those based on the products of Gilead and Abbvie. Surprisingly enough, Gilead did not participate in this tender while Abbvie offered their product with no substantial price rebate thus leading to the failure of the Tuscan experience in fostering more prescriptions for price rebates. This is probably because of the reluctance of these two pharmaceutical firms to accept local agreements in which the nominal price of their products is explicitly reduced.

On the other hand, at national level both Gilead and Abbvie have accepted a price-volume reimbursement agreement with our national Medicines Agency (AIFA) according to which the drug prices are progressively subjected to very substantial price rebates (up to 80%) as the number of treated patients increases (3-5). One reason why the two companies have accepted this national agreement is that the agreement has been kept confidential, and so the nominal values of these discounted prices have remained unknown and have not become a reference price for other jurisdictions.

In conclusion, to manage sustainability in this field, payers of Western countries and national health systems have different tools of procurement and reimbursement at their disposal, but choosing the best strategy is a difficult task because pharmaceutical firms do not adopt the same policies across different countries and different regions.

References

1. Wilensky G. A New Focus on Prescription Drug Spending. JAMA 2015;314(5):440-441.

2. Brunetto MR, De Luca A, Messori A, Zignego AL. Reducing the price of new hepatitis C drugs in the Tuscany region of Italy. BMJ 2015;350:h3363 doi: 10.1136/bmj.h3363 (Published 24 June 2015), available at http://bmj.com/cgi/content/full/bmj.h3363?ijkey=xYS3zhzXoox8A8t&keytype=ref

3. Messori A. Newest treatments for hepatitis C: how can we manage sustainability? Clin Infect Dis. 2015 Aug pii: civ667. [Epub ahead of print], preprint available at http://www.osservatorioinnovazione.net/papers/cid2015pricing.pdf

4. Messori A. Managing the high cost of innovative drugs: anti-cancer agents vs direct-acting antivirals for hepatitis C (Comment posted 2 April 2015), Ann Intern Med 2015, available at http://annals.org/article.aspx?articleid=2212249#tab

5. Quotidiano Sanità Website. “Epatite C. Pani (Aifa) al Senato: Con accordi Aifa risparmi per 2,5 miliardi in due anni. Ma c’è troppo divario tra le Regioni”. http://www.quotidianosanita.it/governo-e-parlamento/articolo.php?articolo_id=30276 , accessed 30 July 2015

On 2016 Nov 22, Lydia Maniatis commented:

his article exhibits at least four common pathologies of the vision science literature.

First, the authors have adopted what, after Graham, I refer to as the “transparent brain hypothesis.”

Graham (1992), notes that, at a time when neuroscientists thought V1 was pretty much all there was to the visual cortex, many psychophysical experimental results perfectly matched the early descriptions of V1 neural characteristics.

Unfortunately, later evidence showed not only that there were many other hierarchically later levels of processing (V2, etc) but that even the descriptions of V1 receptive fields were overly simplistic.

How then, Graham asks, can we explain the mountains of lovely psychophysical results? Her reply is that, under certain conditions, the brain becomes “transparent” and experience directly reflects V1 activity. Teller (1984) had already described this attitude as the “nothing mucks it up proviso,” which she didn’t think was sound. Here, Kwon et al seem to believe that the use of short line segments in their stimuli causes them to directly tap into the V1 layer. (Discussions of both Teller (1984) and Graham (1992) can be found on PubPeer).

Proponents of this frankly bizarre view need, at the least, to meet the burden of explaining what happens at all the other levels of visual processing, with respect to their phenomenon of interest - bearing in mind that the same V1 receptive field activity that mediates observers' experience of their stimuli underlies, and thus is consistent with, and thus must explain, every aspect of their visual experience.

Second, and relatedly, the authors treat perception as a detection problem, rather than as an indirect, inferential process. They say: “Before providing details of our model, we will summarize some relevant characteristics of the topographical map of area V1, which make it uniquely suited for detecting closed curves on a retinal image.”

The reference to closed curves on a retinal image is unfortunate, since the stimulation of the retina is point stimulation, and curves are not a property of the proximal stimulus, but of the percept. Figure-ground segmentation is not, as I'm sure the authors are well aware, achieved by directly reading off retinal activity, in a process of detection. Any description of “contour detection” that can be developed with respect to the known properties of V1 receptive fields will be too simplistic; as a result, it will be trivial to construct any number of falsifying cases requiring a much broader - both geometrically and theoretically - perspective.

Actually, we don’t even have to try to find a falsifying case, because the authors do it for us: “But how about extracting overlapping and intersecting curves, for example, two elongated ellipses intersecting at 4 points? One of the anonymous Reviewers raised this question. The model, in its present form, does not guarantee that such [overlapping] individual ellipses can be extracted: the ambiguities at X intersections will usually not be resolved correctly…” The literature is filled with ad hoc models of this type, i.e. models that “in their present form” are inadequate to their stated purpose.

In the article highlights, the authors call theirs “the first principled explanation of the center of gravity tendency;” but one could argue that it isn’t very principled to describe a failed hypothesis as having explained anything. Perhaps the model can be fixed so as to handle a couple of overlapping ellipses, but it will almost certainly fail again soon thereafter. Sequential ad hoc fixes probably won’t result in an adequately limber model.

Finally, as is often the case with psychophysical experiments, the number of observers is very small (three), one of whom is an author. We’re told, for good measure, that one of the observers was naïve to the purpose of the experiment. Is this naivete important? If so, then what about the other two observers? If not, then why mention it?

I think what the authors refer to as "contour fragments" can also be considered "contours," i.e. they're homogeneously colored figures with a (closed) contour. That is, the "contour fragment" / "contour" dichotomy is a false one.

On 2015 Sep 21, Toni Mueller commented:

Please note regarding the subcellular fractionation protocol to obtain a synapse-enriched fraction (SYN) as indicated in the Methods and Figure 1: subsequent experiments in our lab have indicated that it is not necessary to combine P1 and P2 prior to triton solubilization. While combining these pellets enhanced the protein yield of the SYN fraction and the triton-insoluble other heavy/intermediate membrane fraction to facilitate western blot sample preparation, the SYN fraction generated by triton solubilization of only P2 appears to have less contamination by nuclear proteins. The steps performed in calculating GABA(A) receptor subunit expression and ratios do account for this potential confound (subunit expression is normalized to an inhibitory synapse marker-gephyrin, a loading control- VCP, and subunit expression in total homogenate), but other researchers and labs measuring proteins with both nuclear and synaptic expression patterns should be aware of this limitation in the fractionation method.

On 2016 Dec 01, Graham Walker commented:

There's an interactive version of the Revised Myeloma International Staging System at MDCalc (and one for the original Myeloma International Staging System as well). Both have supplemental content on how to use them clinically, next steps, and info about the creators.

On 2015 Oct 06, KEVIN BLACK commented:

A point of clarification for Ballard et al's Table 1: Nichols et al 2013 do report a death in the placebo group (1 of 9 subjects allocated to placebo, vs 0 of 15 allocated to olanzapine; see Figure 1 and Results text). We would be glad to share additional subject-specific information about adverse events.