On 2016 May 17, Lydia Maniatis commented:

I have a number of issues with this paper, among them that it refers to "perceptual averaging" as though it actually exists, before admitting, at the very end, that it might not. It's an odd situation, since "perceptual;" can an experience be perceptual in the absence of something being actually perceived. (Who are you going to believe, the investigators or your lying eyes?) Apparently, the authors and others they cite have never personally perceived perceptual averages (if they had there would be little doubt of their perceptual existence), but they have purportedly generated a great deal of evidence that other humans do. If, on the other hand, they mean to refer to some type of blindsight, then it is still the case that the term perceptual is not quite right, since the striking thing about blindsight is precisely the lack of a percept.

I've addressed the averaging issue and its existence in other comments, including a comment here on Bauer (2015) and Solomon, May and Tyler (2016).

Another problem I have is that the authors refer to perception in terms of "signal detection," in which the main problem is what to discard, as though "features" were an intrinsic part of the proximal stimulus pre-organization, and we just have to decide what to get rid of or to "summarise." I've criticised the signal detection model in various comments including one on Allard and Faubert (2014). Such descriptions completely miss the point, which is that "features" such as shape and even color are not simply given in the stimulus, which consists of disconnected photons striking the retina. The failure of contemporary "signal detection" and "statistical summary" proponents to understand the fundamental problems of perception, and the tendency to essentially oversimplify the problems, puts them in the same boat with the structuralists, behaviourists and psychophysicists of yore. Of course, what is not simple are the fussy algorithms that are constantly being fitted and adjusted to the data, but this is just a technical not a conceptual complexity. If these activities had heuristic value, rather than being exercises in post hoc data-fitting, then we wouldn't be in a position where, as Bauer (2015) in a review of related research up to 2000, can state that some people continue to doubt the very existence of the assumed processes, or where the present authors could say pretty much the same thing: "Future work will be required to evaluate such claims fully, determining whether what appear to be the effects of perceptual averages might in the end reduce to the effects of memory for fine details of individual stimuli." Even though he's gotten a lot of flack for it, Helmholtz's "unconscious inferences" are basically a description of unconscious processes. Unperceived percepts, on the other hand, are simply paradoxical.

To the failure of the authors to appreciate the subtle but fundamental problems that the Gestaltists were addressing is added the misrepresentation of the latter, who are described as an early version of the former: "As Haberman and Whitney (2012) have noted, the idea that the visual system extracts summary statistics at the expense of individual features is far from a new one, going back at least as far as the writings of first-generation Gestalt psychologists (e.g., Koffka, 1935, pp. 270, 273). Despite the idea's long history and the recent increase in efforts to understand the statistical representations generated by the visual system, the basic structural mechanisms and functional significance of such summary statistics remain unclear." The reference to Koffka is completely off, as in the relevant passages he's discussing binocular vision and the matching or "summarising" of two retinal images to produce the perception of a single one in depth. The Gestaltists would have been acutely sensitive to the inadequacies and contradictions in the arguments in the current literature. They have extensively argued against past incarnations of the same fallacies.

On 2015 Oct 02, Robert Groom commented:

Thank you for your scholarship and insights related to Gibbon’s first successful open heart procedure with cardiopulmonary bypass. There are many lessons to be learned from this amazing “story behind the story.” Do you think that as a scientific community, we have improved in our ability to accurately characterize medical advances in the peer reviewed published literature? On another note, I admire your humility and your call to clinicians to strive to make scholarly contributions.

On 2015 Oct 06, Tom Kindlon commented:

No objective outcome measures were used

This trial just used subjective outcome measures. Objective outcome measures are important as subjective outcome measures may not translate into objective improvements with graded activity-oriented interventions in Chronic Fatigue Syndrome (CFS) (1-3). Examples of more objective outcome measures that have been used in CFS interventional studies include actigraphy, employment data, disability payments data and exercise testing.

References:

1 Kewley AJ. Does Cognitive Behavioral Therapy or Graded Exercise Therapy Reduce Disability in Chronic Fatigue Syndrome Patients? Objective Measures Are Necessary. Clinical Psychology: Science and Practice 2013:20;321-322.

2 Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg G. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychol Med. 2010:40;1281-7.

3 Kindlon T. Comment on: Exercise therapy for chronic fatigue syndrome. http://www.ncbi.nlm.nih.gov/pubmed/25674924#cm25674924_11779

On 2015 Sep 30, Hiskias Keizer commented:

More studies studying CLA induced fat loss in humans have been published than referred to in this study. The large majority of high quality studies among these(meeting EFSA guidelines)do report fat-loss. However, the minimum time required to find such an effect in group sizes as most often used (10-20)is 12 weeks of daily exposure to CLA. This study had a duration of 8 weeks, and was therefore probably too short for finding a fat-loss effect of CLA in a study with this limited size.

On 2015 Sep 26, Eric Fauman commented:

I applaud the authors for identifying novel genetic associations with metabolites but I disagree with their interpretations and conclusions in several regards.

As tempting as it is to use eQTL data to assign causal genes to SNPs it is frequently seen that SNPs tag expression of unrelated genes as often as they tag the true causal gene for the given trait.

In this study the most obvious example is at rs2066938 where the authors report eQTL associations with 5 egenes (RNF10, MLEC, UNC1198B, CAMKK and COQ5), but not ACADS which is almost certainly the true causal gene, as the authors acknowledge in the text.

At the ARG1 and CRAT loci, other genes have stronger eQTL signals so here too the eQTL data is incomplete.

The ALMS1/NAT8 locus is less clear, but previous authors have assigned this locus to NAT8 given the association with N-acetylornithine and NAT8's presumed acetylation function. The biochemical linkage of N-acetylornithine and arginine in the urea cycle suggests that NAT8 is also the causal gene for this paper. If NAT8 is truly the causal gene, the eQTL data missed it at this locus.

A striking example of the over-reliance on eQTL data in this paper is at the "PPP1R16A" locus which associates with the ratio of aspartic acid to alanine. This SNP is in fact just upstream of GPT which encodes glutamic-pyruvic transaminase, also known as alanine transaminase. GPT is a far more plausible causal gene even though it is not one of the 10 egenes listed for this SNP. Interestingly, there is a coding variant in GPT in reasonable LD with the lead SNP (rs1063739, r2=0.77).

In fact 6 of the loci are linked to coding variants in the most probable causal gene (NAT8, GPT, ACADS, SLC22A16, MCCC1 and CPS1).

Again, it's great to see new SNP-metabolite associations still emerging. However any GWAS interpretation must make use of all biological lines of evidence and not rely only on one or two types of data or analysis.

On 2015 Nov 03, John Tucker commented:

The authors find evidence in support of the unsurprising conclusion that pharmaceutical employees are loathe to criticize their own company's products. I don't think anyone will contest that. They also find that academic authors who have accepted research funding from industry currently or in the past are less likely than those who have not to make critical remarks in the paper abstract, and conclude that this represents the influence of conflict-of-interest.

While the first conclusion is both clear and expected, the second involves an implicit and unproven assumption that those who eschew all contact with industry represent an absolute standard of objectivity against which others can be measured. While one can readily point a finger at research support as a conflict, the same reasoning requires that we ask why others have turned research money down. Is it purely a desire for both the actuality and appearance of complete objectivity? Does it stem from a pre-existing distrust and dislike of for-profit drug development? All we really know is that there are likely systematic differences between those who work with industry to develop drugs and those who keep industry at arms length.

One way to address this question might be to take the list of authors of meta analyses included in this study and assess the extent to which positive/negative assessments of individual drugs correlates with public expressions of positive/negative opinion about the pharmaceutical industry as a whole. I suspect that the observed correlation is high, and that when reading any meta analysis, one should be aware that the subjective and non-transparent processes of study selection and data reduction makes most meta analyses little more than detailed editorials.

On 2015 Sep 25, Jacob H. Hanna commented:

The Hanna group congratulates our friends, colleagues and collaborators on their elaborate, timely and elegant review. We wish to alert the readers to four minor constructive comments and points, which may be useful for future discussions:

1) On page 473 the authors state: “Direct derivation of ground state ES cells from human embryos would be a landmark”

We fully agree with the authors that for any newly established stem cell growth conditions, it is important to show technical feasibility for such conditions to obtain newly derive human ESCs from pre-implantation blastocysts. However, it should be kept in mind that the pluripotent state identity is going to be dictated by the derivation growth conditions and not by whether their source was from the human inner cell mass (ICM). The fact that human ICM cells explanted in conventional/primed FGF2/TGFB1 containing conditions, undergo a dramatic in vitro adaptation process and have yielded only primed human ESCs for the last 20 years (Thomson et al. Science 1998), certainly does not prove that conventional human ESCs are in an ICM-like/naïve state. Similarly, mouse ICM derived cells yield naïve stem cells when derived in mouse stem cell naïve conditions in vitro, or give rise to primed Epiblast Stem Cell (EpiSCs) when explanted and derived under mouse primed pluripotency growth conditions (Hanna et al. Cell Stem Cell 2009, Najm et al. Cell Stem Cell 2011). http://www.cell.com/cell-stem-cell/abstract/S1934-5909(11)00049-X http://www.cell.com/cell-stem-cell/abstract/S1934-5909(09)00169-6

The latter examples constitute a clear cautionary note against using ability for ESC derivation from human ICMs in itself as a validation criterion for endowing naïve-like pluripotent state characteristics in any in vitro expanded stem cell type or condition. In fact, we modestly believe that applying such a criterion to annotate the state of an in vitro captured pluripotent cell would be wrong. ESC derivations from human ICM are only technical validations for new growth condition capabilities and applications, and can yield naive or primed pluripotent cells depending on the conditions applied.

2) Less importantly, on page 473 the authors indicate: “Nonetheless, a recent study described altered primate PS cells that can incorporate into host embryos and develop into chimaeric fetuses with low-grade contribution to all three germ layers and early germ cell progenitors57. As in mice, high-grade contribution and germline transmission remain as more stringent tests to demonstrate naive pluripotency in primate ES cells“

We find it unfortunate that this review fails to clearly indicate that the “altered” primate PS cells used in Reference 57 and capable of generating monkey chimeric fetuses for the first time ever, were expanded in negligibly “altered” NHSM conditions previously described by our group in Gafni et al. Nature 2013 (Reference 52). For clarification, the latter fact is clearly indicated by the author in Chen et al. Cell Stem Cell 2015 (Reference 57). http://www.cell.com/cell-stem-cell/abstract/S1934-5909(15)00264-7

3) On Page 473 the authors indicate: “More compelling evidence for cross-species blastocyst chimaerism has been reported following injection of primate naive iPS cells into mouse blastocysts, leading to clonal contribution to solid tissues56.”

We find it unfortunate that this review fails again to indicate that the naïve monkey iPSCs used in Reference 56, and capable of generating cross-species chimerism after micro-injection in mouse blastocysts, were also expanded in presumably “altered” NHSM conditions previously described by our group in Gafni et al. Nature 2013 (Reference 52). Instead of providing exogenous low-dose TGFB1 as devised in NHSM conditions, Fang et al. Cell Stem Cell 2014 (Reference 52) simply used mouse embryonic feeder cells that are known to secrete other TGF family members ligands (including Activin A), and similarly substitute for TGFβ1 in supporting pluripotent stem cell expansion. http://www.cell.com/cell-stem-cell/abstract/S1934-5909(14)00395-6

4) At the end of Page 471 the authors indicate: “The observation that naive cells tolerate depletion of epigenetic regulators supports the concept of naive pluripotency as a configuration with a reduced requirement for epigenetic repression compared to primed PS cells and somatic cells.”

The authors did not provide a citation for the the first and only study thus far conducting side by side comparison on mouse naive and primed cells and showing for the first time opposing tolerance of epigenetic repressor depletion in naive and primed cells from the same species, which was conduced by our group (Geula et al. Science. 2015 Feb 27;347(6225):1002-6. doi: 10.1126/science.1261417).

We had indicated in the discussion by Geula et al Science 2015: “The fact that murine naïve cells, rather than primed cells, are tolerant to depletion of epigenetic and transcriptional repressors supports the concept of naïve pluripotency as a configuration with a relatively minimal requirement for epigenetic repression (in comparison to primed pluripotent and somatic cells)”.

Relevant data and discussion backing these original conclusions are presented in Figure 1 http://www.sciencemag.org/content/347/6225/1002.long, Supplementary Page 17 and Figure S1 http://www.sciencemag.org/content/suppl/2014/12/30/science.1261417.DC1/1261417.Guela.SM.revision1.pdf of Geula et al Science 2015.

With great appreciation,

Jacob (Yaqub) Hanna M.D. Ph.D.

Department of Molecular Genetics

Weizmann Institute of Science

Email: jacob.hanna@weizmann.ac.il

Lab website: http://hannalabweb.weizmann.ac.il/

Twitter: @Jacob_Hanna

On 2017 May 03, Andrea Messori commented:

Please refer to the Letters' section of the Journal.

Presenting the results of a pharmacoeconomic study: incremental cost-effectiveness ratio vs net monetary benefit

Andrea Messori and Sabrina Trippoli

HTA Unit, ESTAR Regional Health Service, 50135 Firenze (Italy)

The article by Wouters and colleagues (1) presents an exhaustive overview on how QALYs can be used in cost-effectiveness analysis. In this framework, the authors also mention the incremental cost-effectiveness ratio (ICER), which is the parameter typically employed to express the results of a cost-effectiveness study. The article, however, does not discuss the net monetary benefit (NMB), which is another parameter employed to express the results of a cost-effectiveness study.

The incremental cost (deltaC) and the incremental effectiveness (deltaE) are the two main parameters of pharmacoeconomics and cost-effectiveness analysis, along with the willingness-to-pay threshold (lambda). The decision rule (e.g. in the case of a favourable pharmacoeconomic result) is (deltaC/deltaE)<lambda (Equation 1), if based on the ICER, or (deltaE x lambda - deltaC) > 0 (Equation 2), if based on the NMB. Likewise, an unfavourable pharmacoeconomic result is when (deltaC/deltaE)>lambda or when (deltaE x lambda - deltaC) < 0; NMB is defined as deltaE x lambda - deltaC, while ICER is defined as deltaC/deltaE. Despite its apparent complexity, most part of pharmacoeconomic methodology is described by the two simple equations reported above (i.e. Equations 1 and 2), but whether the ICER or the NMB is the best parameter for the purposes of pharmacoeconomic decision-making remains on open question.

The study by Cowper et al evaluating new versus old oral anticoagulants in patients with atrial fibrillation (2) is a typical ICER-based cost-effectiveness analysis in which the ICER of apixaban versus warfarin is compared against a willingness-to-pay threshold. This analysis can be taken as an example for comparing ICER vs NMB.

In one of the base-case analyses of the study by Cowper et al, QALYs per patient were 7.94 for apixaban and 7.54 for warfarin, while pharmacological costs per patient were $22,934 and $4,392, respectively. These data yielded, for apixaban versus warfarin, an ICER of $46,355 per QALY gained, a value that remains within the willingness-to-pay threshold of $50,000 per QALY gained and is therefore considered favourable (or “high value care”). As pointed our by Hlatky (3), in interpreting a specific ICER value, more than a single willingness-to-pay threshold is frequently considered (e.g. the threshold between $50,000 and $150,000 or the threshold above $150,000), and this allows us to better understand a pharmacoeconomic result expressed on the basis of an ICER.

In the methodology of pharmacoeconomics, the net monetary benefit (NMB) plays a role similar to that of ICER, but some differences are important.

Firstly, the ICER –by definition- has always an incremental nature and consequently the absolute cost-effectiveness ratio (calculated for a single treatment in the absence of any comparison) makes little sense and, for this reason, is rarely employed. In contrast, the NMB can be calculated for a single treatment in the absence of any comparison (absolute NMB) or can conversely be calculated as an incremental parameter [according to the equation: (incremental NMB) = (incremental QALYs per patient) x (willingness-to-pay threshold) – (incremental cost per patient)]. Another feature of NMB is that the incremental NMB for the comparison of A vs B can be estimated as the absolute NMB calculated for A minus the absolute NMB calculated for B. In this sequence of calculations, calculating the absolute NMB makes sense because the absolute NMB (separately calculated for the experimental treatment and for the control treatment) represents an intermediate step in the calculation of the incremental NMB (Table 1)

The values of absolute NMB for apixaban and warfarin (Table 1) are, respectively, 374,066 and $372,608 per patient (calculated according to Equation 2). Hence, the incremental NMB for apixaban vs warfarin is simply the difference of the above two values, i.e. $1,458 per patient.

References

[1] Wouters OJ, Naci H, Samani NJ. QALYs in cost-effectiveness analysis: an overview for cardiologists. Heart. 2015 Dec;101(23):1868-73.

[2] Cowper PA, Sheng S, Lopes RD, Anstrom KJ, Stafford JA, Davidson-Ray L, Al-Khatib SM, Ansell J, Dorian P, Husted S, McMurray JJ, Steg PG, Alexander JH, Wallentin L, Granger CB, Mark DB. Economic Analysis of Apixaban Therapy for Patients With Atrial Fibrillation From a US Perspective: Results From the ARISTOTLE Randomized Clinical Trial. JAMA Cardiol. 2017 Mar 29. doi:10.1001/jamacardio.2017.0065. [Epub ahead of print]

[3] Hlatky MA. Are Novel Anticoagulants Worth Their Cost? JAMA Cardiol. 2017 Mar 29. doi: 10.1001/jamacardio.2017.0126. [Epub ahead of print]

Table 1. Cost-effectiveness of apixaban vs warfarin in atrial fibrillation: basecase analysis reported by Cowper et al.(2)

a) STARTING VALUES

apixaban: QALYs per patient = 7.94, cost per patient = $22,934

warfarin: QALYs per patient = 7.54, cost per patient = $4,392

willingness-to-pay threshold = $50,000/QALY

b) PHARMACOECONOMIC PARAMETERS

ICER = ($22,934 - $ 4,392) / (7.94 – 7.54) = 18,542 / 0.40 = $46,355/QALY

absolute NMB for apixaban = 7.94 x $50,000 – $22,934 = $374,066

absolute NMB for warfarin = 7.54 x $50,000 – $4,392 = $372,608

incremental NMB = absolute NMB for apixaban - absolute NMB for warfarin = $374,066 - $372,608 = $1,458

c) INTERPRETATION

ICER (equal to $46,355/QALY): favourable because <$50,000/QALY

incremental NMB (equal to $1,458 per patient): favourable because >0

On 2016 Mar 24, Jan Tunér commented:

The 20 J stated here rather appears to be 20 J/cm2, as this fluence has been used in two previous studies by these authors.

On 2017 Jul 04, Randi Pechacek commented:

Alex Alexiev mentioned this article on microBEnet while describing the urban microbiome.

On 2016 Oct 18, Erick H Turner commented:

The ClinicalTrials.gov entry for this study (https://www.clinicaltrials.gov/ct2/show/record/NCT01584440?term=NCT01584440&rank=1) contains the original registration information, but as of October 2016, study results have still not been posted. Credibility of the published results would be enhanced by doing so.

On 2015 Nov 30, Geriatric Medicine Journal Club commented:

This article was critically appraised at the November 2015 Geriatric Medicine Journal Club (follow #GeriMedJC on Twitter). Concerns discussed included that the trial was funded by pharma, last-observation carried forward is not a recommended method for imputing missing data in dementia trials, biases introduced by the choice of rescue medication (Lorazepam), agitation in this trial lumped together several different behaviours (verbal aggression, physical aggression, and non-aggressive activity), and mode of action of the medications. Nevertheless, the findings are interesting but perhaps not quite ready for wide adoption. A transcript of the full discussion can be found at: http://gerimedjc.blogspot.com/2015/11/dextromethorphan-quinidine-not-quite.html?spref=tw

On 2017 Apr 27, Randi Pechacek commented:

Trent Northen, one of the authors of this paper, wrote an interesting blog post on microBEnet giving some background behind this research.

On 2015 Oct 03, Robert Groom commented:

The author raises an important question about use of candidate demographic data and exam results to identify opportunities to improve the education of perfusionists. Clearly there is a moral obligation to use information to inform and refine perfusion education.

On 2015 Sep 28, Bill Cayley commented:

A good piece of research suggesting that sometimes "less" is "more": https://lessismoreebm.wordpress.com/2015/09/28/comparative-effectiveness-of-generic-versus-brand-name-antiepileptic-medications/

On 2015 Sep 26, Lydia Maniatis commented:

None

On 2015 Sep 25, James Hadfield commented:

This appears to be a carbon copy of http://www.ncbi.nlm.nih.gov/pubmed/25278450 and may need investigating

On 2016 Jun 12, John Duffy commented:

Kari Poikolainen and I sent a letter to the Lancet about this paper. It was not published. The content is as follows

The authors report no association between level of alcohol intake and a composite health measure. They write that "In summary, our study shows that current drinking is not associated with a net health benefit." What they fail to mention is that their study also "shows" that current drinking is not associated with net health harm. The interpretation, "sufficient commonalities to support global health strategies and national initiatives to reduce harmful alcohol use" does not follow from their study, which taken on its own suggests that there is no need for such initiatives. However there are a number of methodological problems with their data and analysis. The follow-up time (median 4.3 yrs) was short and the proportion of alcohol consumers (33%) small. There is a strong implication of overfitting, and in the stratified analyses in particular a shortage of degrees of freedom. Since a large body of earlier well-conducted studies shows evidence of both benefits and harms it would be inappropriate to base conclusions on this work.

On 2015 Sep 26, David Keller commented:

Dose-response relationship observed between concentration of ingested alcohol and cancer rate

This study reported that "the risk for cancer was 38% higher in wine drinkers than in never drinkers (HR, 1.38; 95% CI, 1.05 - 1.81), 69% higher in spirit drinkers (HR, 1.69; 95% CI, 1.26 - 2.26), and 20% higher in beer drinkers (HR, 1.20; 95% CI, 0.91 - 1.57)." These 3 data points demonstrate a dose-response curve relating the concentration of ingested alcohol with increased risk of cancer. For cardiovascular (CV) disease, it is the amount of alcohol consumed per day, not its concentration, which is known to be the relevant modifiable risk factor. The maximum CV benefits are associated with consumption of around 2 standard servings per day of alcohol of any concentration.

Note that the 95 percent confidence interval for the Harm Ratio due to drinking beer spanned 1.0, and thus beer did not raise the risk for cancer to a statistically significant degree.

5 percent Beer.....HR = 1.2 non-significant

12 percent Wine.....HR = 1.38

40 percent Spirits..HR = 1.69

Dose-Response Between Alcohol Concentration and Harm Ratio for Cancers

Thus, drinking beer, which, in moderation, is associated with significant cardiovascular (CV) benefits, is not associated with any statistically significant increase in cancers, as are wine and spirits, presumably due to beer's lower alcohol concentration.

This study did not report separate findings for high-volume and low-volume beer drinkers for its risk-benefit analyses. If beer drinkers who limited themselves to 2 servings per day had been analyzed separately, might the harm ratio for incident cancers have been even lower? Might a reduction in cancer incidence perhaps have been observed? At what volume of daily alcohol ingestion in the form of beer (the least carcinogenic alcoholic beverage) was the greatest improvement seen in the composite outcome of cancer and CV disease? Does the optimum amount of beer per day differ for the composite outcome versus the optimal 2 servings per day associated purely with optimal CV outcomes?

The main purpose of studies such as this should be to result in actionable information. Telling us that low economic level countries exhibit different outcomes associated with the consumption of alcohol does little good unless we are given clues as to what alcohol consumers in poor countries might be doing differently than their counterparts in richer nations. Are they drinking higher concentrations of alcohol, greater volumes, both, or is their nutritional status or some other factor implicated?

On 2016 Mar 10, Lydia Maniatis commented:

I'm interleaving my comments to Hoffman's last reply below:

Hoffman: The interface theory predicts that realism is false, and thus that Bell correlations exist. Were experiments to demonstrate that Bell correlations do not exist, then the interface theory would be falsified. Thus the interface theory is an empirical theory, and its prediction is experimentally testable.

LMM: There's a difference between constructing a theory and simply making a loosely argued assertion. Your “theory” doesn't have the logical consistency, respect for facts and specificity to actually predict Bell's correlations. Many people have said the real world doesn't exist, but that doesn't make them physicists, it doesn't make sense to say that they had in effect predicted Bell's correlations. The variables you use are much too vague to justify predictions at any level of specificity, let alone that required at the level of quantum physics.

Hoffman: The remaining experimental loopholes in tests of Bell correlations that Howard Wiseman cites in his paper “Physics: Bell’s theorem still reverberates” are the following: “… they lacked large separations and fast switching of the settings, opening the ‘separation loophole’: information about the detector setting for one photon could have propagated, at light speed, to the other detector, and affected its outcome.” These loopholes are now closed. Bell correlation is confirmed. See Hensen et al., 2015, “Loophole-free Bell inequality violation using electron spins separated by 1.3 kilometres”, Nature, 526, 682-686.

LMM: And there's an even more recent one. However, as I reread Wiseman's article, it is those who (unlike you) don't agree with the Bell's notion of hidden variables (b/c they are empirically inaccessible) need to forgo realism. They (according to Wiseman) must either conclude that some events are correlated for no reason or accept Bell's metaphysical hidden variables. So it's not even clear that Bell's supporters are not realists.

At this point, it seems that physicists are in a very difficult place, caught between two paradoxical options. This is the point at which ground-breaking new theories sometimes emerge. I don't think vision science has a lot to say here.

The fact remains that there are principles of visual perception (and of mechanics, to take an example from physics) that are close enough to the truth in our environment to make useful predictions, lead to useful applications, and to lead to ever more useful theories. That's the world that science lives in. You say this world doesn't exist, yet your story is full of assumptions about that world (natural selection, etc). All I'm saying is, you can't have it both ways.

On 2016 Mar 05, Donald D Hoffman commented:

The interface theory predicts that realism is false, and thus that Bell correlations exist. Were experiments to demonstrate that Bell correlations do not exist, then the interface theory would be falsified. Thus the interface theory is an empirical theory, and its prediction is experimentally testable.

The remaining experimental loopholes in tests of Bell correlations that Howard Wiseman cites in his paper “Physics: Bell’s theorem still reverberates” are the following: “… they lacked large separations and fast switching of the settings, opening the ‘separation loophole’: information about the detector setting for one photon could have propagated, at light speed, to the other detector, and affected its outcome.”

These loopholes are now closed. Bell correlation is confirmed. See Hensen et al., 2015, “Loophole-free Bell inequality violation using electron spins separated by 1.3 kilometres”, Nature, 526, 682-686.

When Wiseman states that “Most physicists are localists” that means that most physicists give up realism. You simply misunderstood him.

On 2016 Feb 17, Lydia Maniatis commented:

This is the status of Bell's theorem in the physics community:

Physics: Bell’s theorem still reverberates Howard Wiseman 19 June 2014

"...But 50 years on, the experimental verifications of these quantum correlations still have ‘loopholes’, and scientists and philosophers still dispute exactly what the theorem states.

...As of now, there are no loophole-free Bell experiments.

...Most physicists are localists: they recognize the two options but choose the first, because hidden variables are, by definition, empirically inaccessible [i.e. outside the realm of empirical science]. Quantum information scientists embrace irreducible randomness as a resource for secure cryptography3. Other physicists and philosophers (the ‘non-localist camp’) dispute that there are two options, and insist that Bell’s theorem mandates non-locality4."

Apparently perception scientists are supposed to side with the minority opinion in physics on an issue of extraordinary complexity, and in turn to figure out what this implies for their discipline. Meanwhile, physics goes on as usual.

Btw, the online descriptions of Bell's theorem often include the term "metaphysics." Again, we are not dealing with empirical science, i.e. testable claims.

The only apparent relevance of the metaphysical claims being put forth as the "interface theory" is that the fundamental assumptions that enable empirical science are false. Yet persistent and irresolvable metaphysical problems with "reality" haven't stopped scientific progress before.

With respect to "H1 and H2," the framing of the questions is paradoxical. What is "it" and what is doing the perceiving? How does the perceiver come into existence? What if no one perceives the perceiver? What if there were no perceivers?

In general, science can never prove that the world exists beyond perception, but it acts as though it could, and that seems to lead to some practical benefits.

On 2015 Dec 10, Donald D Hoffman commented:

Consider two hypotheses.

H1: When it is not perceived, an object has a definite value for each of its dynamical physical properties.

H2: When it is not perceived, an object does not have a definite value for each of its dynamical physical properties.

H1 is widely accepted by vision researchers. Is H1 falsifiable?

If yes: There is an experiment that can support H1 or its negation, viz., H2. Thus H1 is falsifiable iff H2 is falsifiable.

If no: H1 and H2 are both outside empirical science.

In fact H1 and H2 are both falsifiable, and within empirical science. Understanding this nontrivial fact was an intellectual achievement of John Stewart Bell, published in 1964.

Below are published papers presenting empirical tests of the falsifiable hypothesis H2. The results in each case support H2, and thus support the interface theory of perception. A gentle introduction to this literature, and to the remarkable theorem on which it relies, is the paper by Mermin. These experiments, and the theory behind them, are among the greatest achievements of modern science.

Ansmann, M., Wang, H., Bialczak, R. C., Hofheinz, M., Lucero, E., Neeley, M., . . . Martinis, J. M. (2009). Violation of Bell’s inequality in Josephson phase qubits. Nature, 461, 504–506. doi:10.1038/ nature08363

Bell, J. S. (1964). On the Einstein Podolsky Rosen Paradox. Physics 1 (3): 195–200.

Cabello, A., Estebaranz, J. M., & García-Alcaine, G. (1996). Bell– Kochen–Specker theorem: A proof with 18 vectors. Physics Letters A, 212, 183. doi:10.1016/0375-9601(96)00134-X

Giustina, M., Mech, A., Ramelow, S., Wittmann, B., Kofler, J., Beyer, J., . . . Zeilinger, A. (2013). Bell violation using entangled photons with- out the fair-sampling assumption. Nature, 497, 227–230. doi:10.1038/nature12012

Mermin, N. D. (1985). Is the moon there when nobody looks? Reality and the quantum theory. Physics Today, 38(4), 38–47. doi:10.1063/1. 880968

Pan, J.-W., Bouwmeester, D., Daniell, M., Weinfurter, H., & Zeilinger, A. (2000). Experimental test of quantum nonlocality in three-photon GHZ entanglement. Nature, 403, 515–519.

Rowe, M. A., Kielpinski, D., Meyer, V., Sackett, C. A., Itano, W. M., Monroe, C., & Wineland, D. J. (2001). Experimental violation of a Bell’s inequality with efficient detection. Nature, 409, 791–794.

Salart, D., Baas, A., van Houwelingen, J. A. W., Gisin, N., & Zbinden, H. (2008). Spacelike separation in a Bell test assuming gravitationally induced collapses. Physical Review Letters, 100, 220404.

Weihs, G., Jennewein, T., Simon, C., Weinfurter, H., & Zeilinger, A. (1998). Violation of Bell’s inequality under strict Einstein locality conditions. Physical Review Letters, 81, 5039.

On 2015 Dec 02, Lydia Maniatis commented:

Very brief summary: The "falsifiable prediction" that “No object has a position, or any other physical property, when it is not perceived” cannot be tested except by perceiving some objective property of a part of the world (e.g. the absence of an object)...while we are not perceiving it.

It's like turning around really quickly to check if the world behind you disappears when you're not looking at it. Whimsical, but not very scientific.

On 2015 Nov 24, Lydia Maniatis commented:

The authors' proposal NOT FALSIFIABLE (i.e. it is not testable, i.e. it does not belong to empirical science), for a number of reasons.

In addressing the falsifiability question explicitly, Hoffman et al say that “if either [of two predictions – see below] is false, then the interface theory is false” (p. 1501).

The “predictions” being referred to are the following: 1) “No object has a position, or any other physical property, when it is not perceived.” 2) “No object has any causal powers.

The problem should already be obvious. The “interface theory” says that we cannot perceive the state of the objective world, no way, no how – that even space-time is not a property of the physical world. Therefore, on what basis can we ever corroborate any claims regarding position/physical properties of objects? The same goes for “prediction” number 2. We would have to be able to say "Look, this here object HAS a position and physical properties x, y, z EVEN WHEN NOT PERCEIVED! I can't perceive it or any other object, but I can somehow access this information!" With no access to the physical world, we can provide no proof possible about claims regarding the physical world. The assumptions of the “theory,” in other words mean that PROVING IT WOULD BE TANTAMOUNT TO DISPROVING IT.

Another reason that the theory is unfalsifiable is that its computations are based on statements such as: W = a set of world states. How can we ever substitute any kind of numerical value here? I suggest that the “precision” the authors claim for their proposal collapses with this impossibility. (Perhaps as a non-mathematician I'm missing something...).

Also, as an analogy, the interface theory fails to make the case (and the analogy is pretty much the whole case). The claim is that, just as the e.g. square, blue file icon on the desktop doesn't correspond in its properties to the underlying electronics, so percepts don't correspond in any way to physical objects. But our perception of the blue square presumably corresponds to the spectral properties of the light, and the luminance relationships, of the surface of the computer. Hoffman et al can claim that our perception of the properties of the icon are not veridical, but not using the lack of correspondence between icon and electronics as evidence. No one claims that because we can't accurately perceive the shape of a tree because we can't also see the interior, back, atoms that compose it...

Basing broad, unfalsifiable statements such as those being made in this article on a particular, crude similation is not credible. The “non-linearity” assumptions are said to be frequent but not universal, and even the examples given don't hold up. We are told that “not enough water and one dies of thirst; too much and one drowns; somewhere in between is just right” (p. 1506). This is no argument against the accurate (linear) perception of quantity – seeing a lot of water doesn't mean we actually have to drink it all. We might want to decide to swim in it.

There's just no reason to claim that the refined optical structures and responses of organisms have no purpose in using reflected light to infer and represent the world veridically. It's absurd, and leads to absurdity and, again, unfalsifiable claims. There is absolutely no special relationship between this “theory” and the theory of evolution by natural selection; the implication that a fitness criterion is opposite to a veridical (as veridical as possible and necessary) outcome is unfounded (and, again, unfalsifiable). In fact, the normal assumptions about structure and function (based on perceived structures) are more consistent with evolutionary discussions (they are actually required to make such discussions rational).

The claim, in his comment on the article, by the Editor-in-Chief of this journal that “Our measurements...are species-specific and we should worry if they form the starting point for physical theories” (Hickok (2015, p. 1478) just recycles an old philosophical conundrum that is irrelevant to scientific progress. The “worry” cannot be resolved by science but also cannot, in principle and in practice, affect, let alone “be the future of the science of the mind” (p. 1479) - or of any science (otherwise physicists would also have a problem). It just takes us round in circles for the millionth time. The focus on quantum physics is silly, implying as it does that progress in perception depends on an understanding of the unsolved mysteries of that discipline.

On 2015 Nov 11, Lydia Maniatis commented:

Feldman seems unaware of a fundamental principle of perception. His error (can you spot it?) is contained in the following statements:

“What we can say, often, is whether estimates from distinct perceptual systems “agree.” When I reach out and grasp the banana, I may find that it feels the same size as it looked. This is extremely useful, because it means that the perceptual system is coherent: estimates drawn from different sorts of evidence agree.”

In fact, there is a fundamental asymmetry between our visual sense and our body senses. The body senses defer to vision – what vision says, goes. With respect to haptics, this is described as “visual capture.” In the relevant experiment, a subject observes a straight rod through a distorting lens that makes it appear curved. It feels curved.

In the tilted room, our eyes tell us we're tilted, even though our body senses tell us (or should tell us) that we're upright. We feel tilted.

In the rubber hand illusion, our eyes tell us that the rubber hand is being tickled in the same rhythm as we are experiencing tickling on our real hand. The experience is percieved as originating in the hand that we see, not the hand that is actually doing the feeling.

Thus, when this false argument is used to promote the “Bayesian inference” brand, it is moot:

“Not incidentally, this is the central principle underlying Bayesian inference, which Dennis Lindley (2006) paraphrased as (his emphasis) “BE COHERENT.”

In the case of perception, the “central principle' seems to be resting on a false assumption.

Coherence is also a central principle of scientific argument. It is impossible, in this brief report, for a reader to evaluate Feldman's remaining claims for “B'ian inference.” For a critical analysis of very sloppy arguments, please see my PubPeer comment on Feldman's (2015) “Bayesian models of perceptual organization.”

On 2015 Nov 17, Lydia Maniatis commented:

Anderson concludes his critical commentary on Hoffman, Singh and Prakash's (2015) "Interface theory of perception" with the following remark:

"But remarkable theories require remarkable evidence, and there is currently insufficient evidence to take the metaphysical leap proposed by interface theory."

In fact, as I observe in a recent editorial (http://pec.sagepub.com/content/44/10/1149.full) critical of the "anti-realism" of Hoffman and others, the position is not amenable to evidence; any appeal to evidence in favour of the anti-realist position is paradoxical.

If you say that the features of the perceived world bear no correlation, no connection to the real world, then you cannot EVER produce evidence in support of this position. You cannot prove such a position, because you can't compare the "real world" with your (supposed) wholly inaccessible-to-perception-world. As it is impossible, Hoffman cannot be not doing it, even when he is claiming that he is. He is free to believe anything he wants, but the issue is outside of empirical science.

(I will follow this up with a critique of Hoffman, Singh and Prakash (2105) but wanted to make this quick point.)

On 2015 Oct 29, Dmitri Graifer commented:

I have read with great interest this excellent paper. Regretfully, I found that the authors cite earlier studies in somewhat careless way. First, when mentioning biochemical data on eRF1 motifs involved in stop codon recognition, they ignore data obtained by site-directed cross-linking, which for the first time directly demonstrated that one of the mentioned motifs, GTS, is actually located close to the purines of the A site bound stop codon in the mammalian ribosomal termination complexes (Bulygin et al., 2010, RNA). In this regard, the authors cite only the later paper by Wong et al. (Nucleic Acids Res., 2012) as ref. 25, where a suggestion on implication of this motif in stop codon recognition has been made based on the application of a model ribosome-free system but this paper does not contain any original evidence for the proximity of GTS motif to the stop codon bound at the ribosomal A site. Besides, I have found an “empty” reference 52 (Blanchet et al., Nucleic Acids Res., 2015), which does not contain information on contacts of Arg residues in eRF1 positions 65 and 68 with rRNA as one could expect from the phrase “…and the following Arg65 and Arg68 interact with ribosomal RNA (52)”. In particular, in the paper by Blanchet et al., 2015 (ref. 52) Arg68 is not mentioned at all, and interaction of Arg65 with the rRNA is only hypothesized in the middle part of the penultimate paragraph of the Discussion but it is not shown. I guess that inappropriate referencing corrupts really existing priority in the field and gives distorted ideas on the actually obtained scientific results.

On 2015 Nov 13, Lydia Maniatis commented:

Brainard and colleagues have apparently been using the practical-definition-free term "naturalistic" for at least 18 years, as indicated by the article titled: "Color constancy in the nearly natural image. 1. Asymmetric matches. (Brainard, D. H., Brunt, W. A, & Speigle, J. M. (1997), Journal of the Optical Society of America. A, Optics, Image Science, and l'ision, 14, 2091-2110.)

Without stimulus/variable control, questions cannot be framed in a meaningful, clear way. To say that we are controlling for "degree of naturalism" is clearly not a viable option. Not to mention that, as noted above, proving that color constancy is pretty good in "natural" conditions is not interesting - if it weren't, we wouldn't even be asking the question. The question for scientists is, how is it achieved? Refining the answers to this question requires that we posit the role of specific factors, and this means clarifying what features of our stimulus are being manipulated. Again, "degree of naturalness" is a non-starter, and if investigators want to use it, they need to be clear about what they mean by it, and what features are under their control, and what features are NOT under their control, and might affect their results.

In the present study, not even the factor supposedly being manipulated - illumination - was under the investigators' control (see above). They are taking a graphics package's assumptions about what will be perceived as the color of the illumination at face value.

On 2015 Nov 03, Lydia Maniatis commented:

What is the purpose of the data collected in this study? Assuming there is a purpose, are its methods consistent with achieving it?

The title, “Color constancy in a naturalistic, goal-directed task,” is no guide. The ability of humans to achieve color constancy evolved in the natural environment, on the basis of interaction with that environment. Only goal-directed action could lead to selection for a functional perceptual process. “Natural” conditions are where the process enabling color constancy is expected to be most reliable. A plausible question, then, might be “how accurate can color constancy be at its best (assuming we were interested and could measure this.)” However, this is not what the authors are asking.

The abstract suggests two possible motives. First, we are told that “whether and to what degree constancy supports accurate cross-illumination object selection is not well understood.” It is not clear why we would not expect reliable surface perception under variable lighting to “support object selection.” At any rate, this framing of the question seems to assume good color constancy as the independent variable and “object selection” as the dependent variable. A few sentences later, however, the question seems to change; the authors tell us that they “were able to infer and quantify the degree of color constancy that mediated subjects performance.” Now, what seems to have been under investigation is the degree of subjects' color constancy.

However, the question “How good is a subject's color constancy” is meaningless unless we specify conditions. Under some conditions, it is very good, under others, it fails completely. Much is known about these conditions. Artists, for example, are expert at causing radical failures of color constancy via pictorial impressions of inhomogeneous illumination. Perception scientists sometimes have an inkling of this as well. So what specific conditions did the authors aim to test, and why?

The term used is “naturalistic.” What does this mean? The authors apparently can't say. We are told that they manipulated “complexity” by increasing (relative to a previous experiment) the number of distinct surfaces, but that they “do not have at present a precise technical definition of complexity nor precise ways of relating the concepts of scene complexity and scene naturalness” (p. 19). Evidently, “naturalistic” is being defined as “containing more surfaces than stimuli used in a previous experiment.”

At any rate, the stimuli used are clearly not “naturalistic” in any intuitive sense of the word. The authors themselves describe stimuli in one condition as having “a visually jarring appearance – somewhat reminiscent of an image of an early 1970's discotheque...” (p. 17).

So are the authors at least testing color constancy in a randomly selected stimulus? They are not doing even that. This is because their definition of a color constant selection is not based on subjects selecting an e.g. actually red surface under different illuminations, but on subjects matching a red surface with a surface that, in many cases, is not red (i.e. does not have the same chromaticity as the sample.) The stimuli are images on a computer screen, and the changes of illumination are manipulations that the authors hope, or expect, will correspond to the impression of a change in illumination, e.g. the impression of orange lighting. The theoretical assumptions underlying this expectation are part of the graphics package and are never made explicit. They might need reassessment, as subjects did not always make the predicted “color constant” selections. There was, for example, a deviation from the authors' definition/prediction of veridical choices, which based on deviation from their assumptions, they call a “blue bias.” In other words, subjects sometimes matched a sample surface with one that projected more in the blue part of the spectrum than what the authors defined as a “color constant” match. They “do not have an explanation for this bias” (p. 18). However, whether we are dealing with a bias on the part of subjects' perceptual system (unlikely) or a failed prediction of the authors' prepackaged, implicit color constancy/illumination perception model is impossible to say in this hopelessly confounded study.

I would note, finally, that defining the effect of simultaneous contrast as a “failure of color constancy” constitutes a failure to distinguish between figure/ground contrast and adjustments of color contingent on perceived illumination variation.

On 2015 Sep 22, ANDRES TIRADO-SANCHEZ commented:

Are we near to our Krypton?. World paranoia about future experiments at CERN…again. There is growing concern among world's population about the next experiments of the Large Hadron Collider at CERN related to smash subatomic particles at nearly the speed of light, looming next months (interestingly, if we add one of the most mentioned dates on Google maps ©, it redirects us to CERN). This conCERN has invaded the World Wide Web (both created by CERN) through social networking, news, search engines and very descriptive and witty videos that clearly may misinform, in addition to alarm people about the impending destruction of our planet. Detailed information on what lies ahead the CERN experiments is unknown to most of the population, similar to other “end of the world” events, who have gone without fulfilling what they promised however, such events left a feeling of uncertainty that affected people more susceptible, a feeling of fear of an upcoming uncontrollable black hole, that will lead to our extinction. The dark side of the (I must say, exciting) experiments at CERN, are a source of curiosity, fear and uncertainty. Many renowned physicists are concerned and believe that, at some point, both the amount of energy and the black hole generated will get out of control ending in disaster. In order to calm this (still) small outbreak of paranoia, more information is needed; CERN must open doors to the public, reduce mysticism and establish clear rules and communicate the details of the experiments performed there, the real scope, the real risks and the usefulness for mankind; to the extent that the above mentioned aspects are clarified, the population will not create speculation and feel better knowing that what is currently done at CERN will be for our benefit and not our extinction.

On 2017 Jul 04, Randi Pechacek commented:

Elisabeth Bik examines this paper on a microBEnet blog post.

On 2015 Dec 04, Siddharudha Shivalli commented:

I read the article by Gebre M et al, with a great interest. Authors’ efforts are commendable. In their community based study, authors highlight the lower birth preparedness practice in the study area.

Introduction part of the manuscript should delineate the health problem, research question or the hypothesis in such a way that even reader without subject expertise should be able to understand the present situation and need of the study. Keeping the international readers in the mind, authors should have given a brief explanation of existing programs to address maternal and child health issues, with a special emphasis on birth preparedness, in the introduction. Role of healthcare extension worker in the same should have been delineated. In addition, how the study findings could be useful in streamlining of ongoing strategies to foresee the impact should have been discussed.

The level of birth preparedness should have been reported with 95% confidence intervals (i.e. n=104, 18.3%, 95% CI: 15.3-21.7). Authors used the ‘knowledge of danger signs’ during pregnancy, delivery, and postnatal period as a predictor of birth preparedness. However, in methodology, they should have explicitly mentioned the various danger signs asked to assess the same. In Table 4, authors should have reported row wise percentages to make it clearer. Regression analysis is an excellent way of adjusting the effect of con-founders. However, it is always recommended to assess and report the adequacy of applied regression model. Failure to do so may lead to misleading or incorrect inferences. Although the study sample was relatively large (n=569), a word about R2 (explaining the variance in birth preparedness) of the applied regression model would have been more affirmative.

Nonetheless, I must congratulate the authors for exploring an important public health problem in the study area. No competing interests declared.

On 2017 Apr 25, Jeffrey Ross-Ibarra commented:

In my 2007 paper I find a positive correlation between the number of crossovers per bivalent and genome size. I note that this correlation is weaker than linear, such that genome size increases faster than the number of crossovers per bivalent. This means that measures of recombination that include genome size (e.g. cM/Mb) will decrease with increasing genome size (because the denominator increases faster than the numerator). The findings reported here of a negative correlation between cM/Mb and genome size are thus consistent with those reported in my 2007 paper.

On 2015 Oct 07, Bill Cayley commented:

For adolescents with depression, "less" appears to be "more": https://lessismoreebm.wordpress.com/2015/09/17/restoring-study-329-efficacy-and-harms-of-paroxetine-and-imipramine-in-treatment-of-major-depression-in-adolescence/

On 2015 Sep 29, Carlos Maia commented:

http://www.ncbi.nlm.nih.gov/pubmed/25806553

On 2015 Sep 17, Tom Kindlon commented:

(contd.)

11 National Institute for Health and Clinical Excellence. Chronic fatigue syndrome/myalgic encephalomyelitis (or encephalopathy): diagnosis and management of CFS/ME in adults and children, 2007. http://www.nice.org.uk/guidance/CG53 Accessed September 6, 2015. London: National Institute for Health and Clinical Excellence.

12 White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, et al. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. The Lancet 2011;377:823-36.

13 Kindlon T. PACE Trial - 97% of the participants who didn't have a psychiatric disorder satisfied the definition of M.E. used. https://listserv.nodak.edu/cgi-bin/wa.exe?A2=ind1106A&L=CO-CURE&P=R2764 Accessed: September 6, 2015

14 Ellen Goudsmit on PubMed Commons: http://www.ncbi.nlm.nih.gov/myncbi/ellen m.goudsmit.1/comments/

15 Green CR, Cowan P, Elk R, O'Neil KM, Rasmussen AL. National Institutes of Health Pathways to Prevention Workshop: advancing the research on myalgic encephalomyelitis/chronic fatigue syndrome. Ann Intern Med. 2015; 162:860-5.

16 Haney E, Smith MEB, McDonagh M, Pappas M, Daeges M, Wasson N, et al. Diagnostic methods for myalgic encephalomyelitis/chronic fatigue syndrome: a systematic review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med. 2015; 162:834-40.

17 Smith MEB, Haney E, McDonagh M, Pappas M, Daeges M, Wasson N, et al. Treatment of myalgic encephalomyelitis/chronic fatigue syndrome: a systematic review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med. 2015; 162:841-50.

On 2015 Sep 17, Tom Kindlon commented:

Comment 2 of 2 formally submitted on September 9, 2015 on the full Cochrane review using instructions here: http://www.cochranelibrary.com/help/submitting-comments-on-cochrane-reviews.html

(This is available in one piece at: http://forums.phoenixrising.me/index.php?threads/my-two-detailed-comments-on-the-cochrane-exercise-therapy-for-cfs-review-2015.39801/ )

Second comment:

I have decided to split my comment into two as it was getting very long. This is part two.

Variation in interventions

It would have been useful to have some more information on the “exercise with pacing” intervention tested in the Wallman et al. (2004) trial and how it was distinct from some other exercise interventions tested. The authors say (1): “On days when symptoms are worse, patients should either shorten the session to a time they consider manageable or, if feeling particularly unwell, abandon the session altogether” (p. 143). I don't believe the description given in the review conveys this. In the review, this approach is described as "Exercise with pacing: exercise in which the incremental increase in exercise was personally set." But Wallman et al.’s approach allows patients to decrease as well as increase how much exercise they do on the day. This approach also contrasts with how White (an investigator in two of the trials) has described graded exercise therapy: "if [after increasing the intensity or duration of exercise] there has been an increase in symptoms, or any other adverse effects, they should stay at their current level of exercise for a further week or two, until the symptoms are back to their previous levels" (2). In the PACE Trial manual White co-wrote (3), the GET intervention was guided by the principle that “planned physical activity and not symptoms are used to determine what the participant does” (p. 21); similarly, “it is their planned physical activity, and not their symptoms, that determine what they are asked to do” (p. 20). Compliance data would help us examine which approach patients are actually using: I suspect many patients are in fact doing exercise with pacing even in trials such as the PACE Trial (i.e. when they have increased symptoms, often reducing levels of exercise and sometimes doing no exercise activities at all on that day).

Bimodal versus Likert scoring in Wearden et al. (2010)

I find it odd that the fatigue scores for the Wearden et al. (2010) trial (4) are given in the 0-33 format rather than the 0-11 scoring method. The 0-11 scoring system is what is mentioned as a primary outcome measure in the protocol and is what is reported in the main paper reporting the results (4, 5). It is even what your own report says on p. 44 is the scoring method (“Fatigue Scale, FS; 11 items; each item was scored dichotomously on a 4-point scale [0, 0, 1 or 1]”). This is important because using the scoring method for which you don't report data (0-11), there is no statistically significant difference at the primary outcome point of 70 weeks (5).

Diagnostic criteria

One problem with using these trials as an evidence base, which I don't believe was mentioned, is that all the trials used the Oxford and Fukuda diagnostic criteria (6, 7). Neither of these criteria require patients to have post-exertional malaise (or something similar). Many consider this to be a core symptom of ME/CFS and it is mandatory in most of the other major criteria (8-11). [Aside: The London criteria were assessed in the PACE Trial (12) but they seem to have been operationalised in an unusual way. Ninety seven per cent of the participants who satisfied the (broad) Oxford criteria who didn't have a psychiatric disorder satisfied the definition of M.E. used (13). Ellen Goudsmit, one of the authors of the London criteria, has rejected the way they were used in the PACE Trial (14)]. So this lack of requirement for patients to have post-exertional malaise (or a similar description) means we cannot be sure that the evidence can be generalised to such patients. An independent National Institutes of Health committee this year concluded "continuing to use the Oxford definition may impair progress and cause harm. Therefore, for progress to occur, we recommend that this definition be retired" (15). An Agency for Healthcare Research and Quality review of diagnostic methods this year reached a similar conclusion: "Consensus groups and researchers should consider retiring the Oxford case definition because it differs from the other case definitions and is the least restrictive, probably including individuals with other overlapping conditions” (16). An Agency for Healthcare Research and Quality review of ME/CFS treatments said: "The Oxford CFS case definition is the least restrictive, and its use as entry criteria could have resulted in selection of participants with other fatiguing illnesses or illnesses that resolve spontaneously with time" (17).

Exclusion of some data from analyses due to baseline differences

It seems unfortunate that some data cannot be used due to baseline differences e.g. "Four trials (669 participants) contributed data for evaluation of physical functioning at follow-up (Jason 2007; Powell 2001; Wearden 2010; White 2011). Jason 2007 observed better results among participants in the relaxation group (MD 21.48, 95% CI 5.81 to 37.15). However, results were distorted by large baseline differences in physical functioning between the exercise and relaxation groups (39/100 vs 54/100); therefore we decided not to include these results in the meta-analysis". It would be good if other methods could be investigated (e.g. using baseline levels as covariates) to analyse such data.

Thank you for taking the time to read my comments.

Tom Kindlon

Conflict of Interest statement:

I am a committee member of the Irish ME/CFS Association and do a variety of unpaid work for the Association.

References

1 Wallman KE, Morton AR, Goodman C, Grove R. Exercise prescription for individuals with chronic fatigue syndrome. Med J Aust. 2005;183:142-3.

2 White P. How exercise can help chronic fatigue syndrome. Pulse: 1998. June 20:86-87.

3 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME (Therapist Manual) http://www.pacetrial.org/docs/get-therapist-manual.pdf

4 Wearden AJ, Riste L, Dowrick C, Chew-Graham C, Bentall RP, Morriss RK, Peters S, Dunn G, Richardson G, Lovell K, Powell P. Fatigue Intervention by Nurses Evaluation--the FINE Trial. A randomised controlled trial of nurse led self-help treatment for patients in primary care with chronic fatigue syndrome: study protocol. [ISRCTN74156610]. BMC Med. 2006 Apr 7;4:9.

5 Wearden AJ, Dowrick C, Chew-Graham C, Bentall RP, Morriss RK, Peters S, Riste L, Richardson G, Lovell K, Dunn G; Fatigue Intervention by Nurses Evaluation (FINE) trial writing group and the FINE trial group. Nurse led, home based self help treatment for patients in primary care with chronic fatigue syndrome: randomised controlled trial. BMJ. 2010 Apr 23;340:c1777. doi: 10.1136/bmj.c1777.

6 Sharpe M, Archard L, Banatvala J, Borysiewicz LK, Clare AW, David A, et al. Chronic fatigue syndrome: guidelines for research. Journal of the Royal Society of Medicine 1991;84 (2):118–21.

7 Fukuda K, Straus SE, Hickie I, et al. The chronic fatigue syndrome: A comprehensive approach to its definition and study. Ann Intern Med. 1994; 121: 953‑959.

8 Carruthers BM, Jain AK, De Meirleir KL, et al. Myalgic Encephalomyelitis/chronic fatigue syndrome: Clinical working case definition, diagnostic and treatments protocols. Journal of Chronic Fatigue Syndrome. 2003; 11: 7-115.

9 Carruthers BM, van de Sande MI, De Meirleir KL, et al. Myalgic encephalomyelitis: International Consensus Criteria. J Intern Med. 2011; 270: 327-338.

10 IOM (Institute of Medicine). Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Washington, DC: The National Academies; 2015.

(continues)

On 2015 Sep 17, Tom Kindlon commented:

(continues)

References:

1 Turner L, Boutron I, Hróbjartsson A, Altman DG, Moher D: The evolution of assessing bias in Cochrane systematic reviews of interventions: celebrating methodological contributions of the Cochrane Collaboration. Syst Rev 2013, 2:79.

2 Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry. 2015;2:141-152.

3 White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, et al. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. The Lancet 2011;377:823-36.

4 Kindlon T. Reporting of Harms Associated with Graded Exercise Therapy and Cognitive Behavioural Therapy in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Bull IACFS ME. 2011;19:59-111. http://iacfsme.org/ME-CFS-Primer-Education/Bulletins/BulletinRelatedPages5/Reporting-of-Harms-Associated-with-Graded-Exercise.aspx

5 Lipkin DP, Scriven AJ, Crake T, Poole-Wilson PA (1986). Six minute walking test for assessing exercise capacity in chronic heart failure. British Medical Journal 292, 653-5.

6 Kadikar A, Maurer J, Kesten S. The six-minute walk test: a guide to assessment for lung transplantation. J Heart Lung Transplant. 1997 Mar;16(3):313-9.

7 Friedberg F, Sohl S. Cognitive-behavior therapy in chronic fatigue syndrome: is improvement related to increased physical activity? J Clin Psychol. 2009 Feb 11.

8 Friedberg F. Does graded activity increase activity? A case study of chronic fatigue syndrome. Journal of Behavior Therapy and Experimental Psychiatry, 2002, 33, 3-4, 203-215

9 Results and In-depth Analysis of the 2012 ME Association Patient Survey Examining the Acceptability, Efficacy and Safety of Cognitive Behavioural Therapy, Graded Exercise Therapy and Pacing, as Interventions used as Management Strategies for ME/CFS. Gawcott, England. http://www.meassociation.org.uk/2015/05/23959/ Accessed: September 3, 2015

10 Critical Illness - A Dreadful Experience with Scottish Provident. http://forums.moneysavingexpert.com/showthread.php?t=2356683 Accessed: September 4, 2015

11 McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS One. 2012;7(8):e40808.

12 Rapport d’évaluation (2002-2004) portant sur l’exécution des conventions de rééducation entre le Comité de l’assurance soins de santé (INAMI) et les Centres de référence pour le Syndrome de fatigue chronique (SFC). 2006. http://health.belgium.be/internet2Prd/groups/public/@public/@shc/documents/ie2divers/14926531_fr.pdf (Starts on page 223.) Accessed September 4, 2015 (French language edition)

13 Evaluatierapport (2002-2004) met betrekking tot de uitvoering van de revalidatieovereenkomsten tussen het Comité van de verzekering voor geneeskundige verzorging (ingesteld bij het Rijksinstituut voor Ziekte- en invaliditeitsverzekering) en de Referentiecentra voor het Chronisch vermoeidheidssyndroom (CVS). 2006. http://health.belgium.be/internet2Prd/groups/public/@public/@shc/documents/ie2divers/14926531.pdf (Starts on page 227.) Accessed September 4, 2015 (Dutch language version)

14 Stordeur S, Thiry N, Eyssen M. Chronisch Vermoeidheidssyndroom: diagnose, behandeling en zorgorganisatie. Health Services Research (HSR). Brussel: Federaal Kenniscentrum voor de Gezondheidszorg (KCE); 2008. KCE reports 88A (D/2008/10.273/58) https://kce.fgov.be/sites/default/files/page_documents/d20081027358.pdf Accessed September 4, 2015

15 Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg G. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychol Med. 2010; 40:1281-1287.

16 Van Kessel K, Moss-Morris R, Willoughby, Chalder T, Johnson MH, Robinson E, A randomized controlled trial of cognitive behavior therapy for multiple sclerosis fatigue, Psychosom. Med. 2008; 70:205–213.

17 Powell P. FINE Trial Patient Booklet http://www.fine-trial.net/downloads/Patient PR Manual ver9 Apr05.pdf Accessed September 7, 2015

18 Twisk FNM, Maes M. A review on Cognitive Behavorial Therapy (CBT) and Graded Exercise Therapy (GET) in Myalgic Encephalomyelitis (ME)/Chronic Fatigue Syndrome (CFS): CBT/GET is not only ineffective and not evidence-based, but also potentially harmful for many patients with ME/CFS. Neuro Endocrinol Lett. 2009;30:284-299.

19 Carruthers BM et al. Myalgic Encephalomyelitis – Adult & Paediatric: International Consensus Primer for Medical Practitioners. ISBN 978-0-9739335-3-6 http://www.investinme.org/Documents/Guidelines/Myalgic Encephalomyelitis International Consensus Primer -2012-11-26.pdf Accessed September 5, 2015

20 Twisk FN. Objective Evidence of Post-exertional “Malaise” in Myalgic Encephalomyelitis and Chronic Fatigue Syndrome. J Sports Med Doping Stud 2015. 5:159. doi: 10.4172/2161-0673.1000159

21 Higgins JPT, Green S: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. Table 8.5.d. The Cochrane Collaboration; 2011. http://handbook.cochrane.org/chapter_8/table_8_5_d_criteria_for_judging_risk_of_bias_in_the_risk_of.htm Accessed: September 5, 2015

22 White PD, Sharpe MC, Chalder T, DeCesare JC, Walwyn R; on behalf of the PACE trial group. Protocol for the PACE trial: A randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurology 2007, 7:6 http://www.biomedcentral.com/1471-2377/7/6 Accessed: September 5, 2015

On 2015 Sep 17, Tom Kindlon commented:

(contd.)

Compliance

The review doesn't include any information on compliance. I'm not sure that there is much published information on this but I know there was a measure based on attendance at therapy sessions (which could be conducted over the phone) given for the PACE Trial (3). Ideally, it would be interesting if you could obtain some unpublished data from activity logs, records from heart-rate monitors, and other records to help build up a picture of what exercise was actually performed and the level of compliance. Information on adherence and what exercise was actually done is important in terms of helping clinicians, and indeed patients, to interpret and use the data. I mention patients because patients' own decisions about their behaviour is likely to be affected by the medical information available to them, both within and outside of a supervised programme of graded exercise; unlike with an intervention like a drug, patients can undertake exercise without professional supervision.

"Selective reporting (outcome bias)" and White et al. (2011)

I don't believe that White et al. (2011) (the PACE Trial) (3) should be classed as having a low risk of bias under "Selective reporting (outcome bias)" (Figure 2, page 15). According to the Cochrane Collaboration's tool for assessing risk of bias (21), the category of low risk of bias is for: "The study protocol is available and all of the study’s pre-specified (primary and secondary) outcomes that are of interest in the review have been reported in the pre-specified way". This is not the case in the PACE Trial. The three primary efficacy outcomes can be seen in the published protocol (22). None have been reported in the pre-specified way. The Cochrane Collaboration's tool for assessing risk of bias states that a “high risk” of bias applies if any one of several criteria are met, including that “not all of the study’s pre-specified primary outcomes have been reported” or “one or more primary outcomes is reported using measurements, analysis methods or subsets of the data (e.g. subscales) that were not pre-specified”. In the PACE Trial, the third primary outcome measure (the number of "overall improvers") was never published. Also, the other two primary outcome measures were reported using analysis methods that were not pre-specified (including switching from the bimodal to the Likert scoring method for The Chalder Fatigue Scale, one of the primary outcomes in your review). These facts mean that the “high risk of bias” category should apply.

Thank you for taking the time to read my comments.

Tom Kindlon

Conflict of Interest statement:

I am a committee member of the Irish ME/CFS Association and do a variety of unpaid work for the Association. (continues)

On 2015 Sep 17, Tom Kindlon commented:

Comment 1 of 2 formally submitted on September 9, 2015 on the full Cochrane review using instructions here: http://www.cochranelibrary.com/help/submitting-comments-on-cochrane-reviews.html

(This is available in one piece at: http://forums.phoenixrising.me/index.php?threads/my-two-detailed-comments-on-the-cochrane-exercise-therapy-for-cfs-review-2015.39801/ )

I would first like to thank those involved for their work in preparing this document. Even for those of us who have read the individual Chronic Fatigue Syndrome (CFS) papers it is useful to have the results collated, as well as details regarding the interventions. Also it is interesting to see the results of sensitivity analyses, subgroup analyses, standardised mean differences, etc.

I would like to make a few comments. I’m splitting them into two submissions as the piece had become very long. I’ve added some loose headings to hopefully make it more readable.

Objective measures

The review assessed the studies as having a high risk of bias regarding blinding, since neither participants nor assessors were blinded. Evidence suggests that subjective outcomes are more prone to bias than objective outcomes when there is no blinding (1). It is thus unfortunate that the review concentrated almost exclusively on subjective measures, failing to include results from nearly all the objective outcome measures that have been published with trials. (The exception was health resource use for which you presented follow-up data from one trial).

I hope objective outcome data can be included in a future revision or edition of this review.

Examples of objective outcomes include: exercise testing (work capacity by oxygen consumption); fitness test/step test; the six minute walking test; employment status; and disability payments.

Adding in these results would allow a more rigorous assessment of the effectiveness and relevance of the therapies, their causal mechanisms, therapeutic compliance, and safety.

On exercise testing, for example, in the PACE Trial (the largest trial in the review) there was no improvement in fitness levels as measured by a step test (2). The fitness data contrasts sharply with the many positive results from subjective self-report measures in the trial, so one is left wondering how much the subjective measures reflect reality.

On another exercise test used in the PACE Trial, the 6 minute walk test, there was a small (mean) increase from 312 metres at baseline to 379 metres at 12 months: this was 35.3 metres more than the "passive" control group when adjustments were made. However, the final result of 379 metres remains very poor compared to the more than 600 metres one would expect from healthy people of a similar age and gender make-up (3,4). By comparison, a group with Class III heart failure walked an average of 402 metres (5). A score of less than 400 metres has been suggested as the level at which somebody should be put on a lung transplant list (6). Such information from objective measures helps to add important context to the subjective measures and restraint to the conclusions that can be drawn from them.

Objective data is also needed to check compliance with a therapy. If patients diligently exercised for 12 months one would expect much better results on fitness and exercise testing than the aforementioned results in the PACE Trial. This is important when considering adverse events and safety: such trials may not give us good information on the safety of complying with such interventions if patients haven't actually complied.

Employment and receipt of disability payments are practical objective measures of general functional capacity so data on them would help establish whether patients can actually do more overall or whether they may just be doing, for example, a little more exercise but have substituted that for doing less in other areas (7,8). Also, CFS patients are sometimes pressured by insurance companies into doing graded exercise therapy (GET) programs so it would be useful to have data collated on employment outcomes to see whether pressure can in any way be justified (9,10). In the PACE Trial, there was no significant improvement in employment measures and receipt of disability payments in the GET group (11). Outside the realm of clinical trials, the quantitative and qualitative data in a major (UK) ME Association survey also found that GET didn't lead to higher levels of employment and lower levels of receipt of disability payments on average (9). Also, extensive external audits were performed of Belgian CFS rehabilitation clinics that treated using cognitive behavioural therapy (CBT) and GET. The main reports are in French and Dutch (12,13), with an English summary available (14) that says, "Employment status decreased at the end of the therapy, from an average of 18.3% of a 38h working week, to 14.9% [...] The percentage of patients living from a sickness allowance increased slightly from 54 to 57%." This contrasts with the average improvements reported in the audit for some symptoms like fatigue.

While data on (self-reported) symptoms like fatigue (one of your two primary outcomes) is interesting, arguably more important to patients is improving their overall level of functioning (and again, objective measures are needed here). Being able to work, for example, despite experiencing a certain level of fatigue would likely be more important for many than being unable to work but having slightly lower levels of fatigue.

An example of how reductions in the reported levels of fatigue may not lead to improvements in functioning can be seen in an analysis of three graded activity-oriented CBT therapy interventions for CFS (15). The analysis showed, compared to controls, there were no improvements in overall activity levels as measured by actometers despite improvements in self-reported fatigue (15). Activity in these trials was assessed using actometers. Another study that exemplifies the problem of focusing too much on fatigue scores after behavioural interventions is a study of CBT in multiple sclerosis (MS) patients with “MS fatigue”(16). The study found that following the intervention, patients with MS reported significantly lower (i.e. better) scores on the Chalder Fatigue Scale (0-33 scoring) than those in a healthy, nonfatigued comparison group! This significant difference was maintained at 3 and 6 months’ follow-up. It is difficult to believe that patients with MS fatigue (at baseline) truly subsequently had less fatigue than healthy nonfatigued controls: a much more likely scenario is that undertaking the intervention had led to response biases.

You mention that "many patient charities are opposed to exercise therapy for chronic fatigue syndrome (CFS)". One reason for concern about the way in which exercise programmes are promoted to patients is that they are often based upon models which assume that there is no abnormal physiological response to exercise in the condition, and make unsupported claims to patients. For example, in the FINE trial (Wearden et al., 2010) patient booklet (17), it is boldly asserted that: "Activity or exercise cannot harm you" (p. 49). However, a large number of studies have found abnormal responses to exercise, and the possibility of harm being done simply cannot be excluded on the basis of current evidence (discussed in 4, 18-20)."

(continues)

On 2016 Feb 08, Theo Sanderson commented:

"Briefly, PMA stock solution (20 mM in 20 % dimethyl sulfoxide; Biotium Inc., Hayward, CA) was added at a concentration of 200 mM." It seems likely that the working concentration was in fact 200 μM?

On 2015 Sep 30, Anders von Heijne commented:

This is an important analysis. The only thing missing is the temporal relationship between the debut of the PML-IRIS related encephalitic symptoms and the imaging findings. Do they occur at the same time or are there cases when the imaging findings precede the clinical worsening?

On 2017 Oct 12, Sander Houten commented:

This paper focuses on the role of KLF14 in the regulation of hepatic ApoA-I expression. The authors show that hepatic KLF14 expression is reduced in dyslipidemia mouse models. They show that overexpression of human KLF14 in liver of mice that were fed a high fat diet, increased HDL-cholesterol, cholesterol efflux and ApoA-I expression. Conversely, they show that liver-specific deletion of Klf14 decreased HDL-cholesterol and ApoA-I expression. The authors offer a mechanistic explanation by demonstrating that the ApoA-I promoter contains one KLF14-binding site. Moreover, perhexiline was identified as a KLF14 activator that raised HDL-cholesterol, ApoA-I and cholesterol efflux in vivo and reduced atherosclerosis development in Apoe^−/− mice (Guo Y, 2015). I would like to point out that there is little evidence to support the hypothesis that KLF14 plays an important role in adult mouse liver biology. We found no evidence for expression of KLF14 in adult mouse liver as we were unable to amplify Klf14 cDNA, did not find Klf14 mapped reads in liver RNA sequencing data and found no specific signal upon immunoblotting (Argmann CA, 2017). Our data on the absence of Klf14 expression in liver are consistent with previously published work by others (Parker-Katiraee L, 2007) and publicly available data sources. We also investigated the physiological functions of KLF14 by studying a whole body KO mouse model and focused on the metabolic role of this transcription factor in mice on chow and high fat diet. Our results do not support a major role of hepatic KLF14 in the regulation of ApoA-I and HDL-cholesterol levels (Argmann CA, 2017).

On 2016 Apr 16, Miguel Lopez-Lazaro commented:

Important flaws in the identification of the cells of origin in cancer

This is an extension of this Editorial (Cancer arises from stem cells: opportunities for anticancer drug discovery) and a commentary on the article http://www.ncbi.nlm.nih.gov/pubmed/27044116

Abstract: The cells of origin in cancer are the cells in which carcinogenesis starts, i.e., the normal cells that acquire the first cancer-related hit. These cells should not be confused with the cancer stem cells or tumor-initiating cells, which are malignant cells with the ability to initiate and maintain tumor growth. Recently, many articles published in leading journals claim that they have experimentally demonstrated the cellular origin of several types of cancer. These articles, however, have important flaws that may have led to wrong conclusions. Here I briefly discuss one of the many articles in this field of research to draw attention to these flaws. In addition, I analyze the striking differences in prostate cancer incidence by age to examine the article’s conclusion and to propose that prostate cancer originates in stem cells. The striking variations in cancer risk among tissues also indicate that stem cells are the cells of origin in cancer.

Full text at: Lopez-Lazaro, M. Important flaws in the identification of the cells of origin in cancer. ResearchGate: DOI: 10.13140/RG.2.1.4321.5769; http://dx.doi.org/10.13140/RG.2.1.4321.5769

On 2015 Dec 10, Kenneth Witwer commented:

As a co-author of this meeting report on an RNAi session at the 40th meeting of The Toxicology Forum, I summarized a talk I gave on human health considerations related to environmental RNA molecules. During my talk, I also reflected briefly on a 2014 US Environmental Protection Agency panel and a European Food Safety Authority (EFSA) workshop I had recently taken part in. During the preparation and revision process of this manuscript, EFSA published a workshop report on the EFSA website. I inadvertently did not include a link to this report in the manuscript, so I would like to note that the pdf can be downloaded here. "Selected key outcomes of the workshop" were also described by Ramon M, 2014.

On 2014 Dec 23, Akshaya shetti commented:

can i get the full text of this article?

On 2017 Apr 01, Misha Koksharov commented:

It's interesting whether LREs have a gluconolactonase function. Or whether gluconolactonases have the LRE activity

On 2017 Mar 26, Misha Koksharov commented:

None

On 2017 Jul 04, Randi Pechacek commented:

Matt Perisin, first author of this paper, wrote a "Behind the Paper" blog on microBEnet.

On 2016 Feb 16, Robert Courtney commented:

A study that uses questionable assumptions rather than empirical evidence leads to conclusions that stretch credibility.

Chalder et al. [1] used the “single mediation model” for their methodology, which is explained in detail in a book by MacKinnon [2]. Explaining the methodology MacKinnon says a temporal separation between variables must be observed (i.e. changes in mediating variable must occur before changes in the mediated variable) for a mediation effect to be empirically and robustly established.

Chalder et al. were working to this model and acknowledged that they failed to establish a clear temporal separation between variables, and therefore did not empirically establish a causal mediation effect: “Given the pattern of change in the mediators was similar to the pattern of change in the outcomes it is possible that the variables were affecting each other reciprocally”.

However, despite the lack of robust empirical evidence to support a mediation effect, the investigators concluded that they had established mediation effects, e.g: “Our main finding was that fear avoidance beliefs were the strongest mediator for both CBT and GET.”

The study’s conclusion relied upon an assumption that the investigators’ favoured hypothetical model of illness for ME/CFS has a robust empirical evidence base and is applicable to this study. The hypothesis is based upon the idea that symptoms and disability in ME/CFS are perpetuated by unhelpful or maladaptive illness beliefs, fear, and an avoidance of activity.

However, the prestigious National Academy of Medicine (formerly known as the Institute of Medicine) recently released a comprehensive report [3] into ME/CFS that rejected such a hypothetical model of illness, and unambiguously concluded that ME/CFS does not have a psychological or cognitive-behavioural basis, but is an organic illness that requires biomedical research.

Chalder et al. discussed the possibility that more frequent measurements may have potentially demonstrated a temporal separation between the variables, and therefore a mediation effect. However, this raises the possibility of whether changes in the primary outcome variables (self-report physical function and fatigue) may, in fact, have occurred before changes in the presumed mediator variables. Such an outcome would entirely contradict the investigators’ premature conclusions. According to MacKinnon [2] and Wiedermann et al. [4], unexpected outcomes should not be ruled out.

Chalder et al. concluded that symptoms and physical impairment, in ME/CFS patients, are mediated by activity avoidance and other factors. (e.g. This would mean that a decrease in activity would cause an increase in symptoms.) However, from a common sense point of view, this seems like rather a convoluted conclusion, and it seems more likely that increased symptoms would be the direct cause of activity avoidance in any illness, rather than vice versa. To conclude that activity avoidance causes fatigue (rather than fatigue being a direct cause of activity avoidance), is similar to concluding that a person has flu because they’ve taken a day off work, rather than the obvious conclusion that they’ve taken a day off work because they have flu.

In the case of fatigue, flu-like malaise and other symptoms of ME/CFS, it seems reasonable to consider the possibility that, as the symptoms fluctuate, patients may intuitively or rationally adapt their activity levels according to what is comfortable and safe. i.e. patients reduce activity levels because they are fatigued. The investigators have concluded that patients are fatigued because they have reduced activity levels.

Perhaps patients’ perspectives and insights would help clarify the issues but, unfortunately, patients were not consulted for this study.

References:

1. Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 2015; 2: 141–52.

2. MacKinnon DP. Introduction to Statistical Mediation Analysis. Taylor and Francis: New York 2008.

3. IOM (Institute of Medicine). 2015. Beyond myalgic encephalomyelitis/chronic fatigue syndrome: Redefining an illness. Washington, DC: The National Academies Press. http://iom.nationalacademies.org/Reports/2015/ME-CFS.aspx

4. Wiedermann W, von Eye A. Direction of Effects in Mediation Analysis. Psychol Methods 2015; 20: 221-44.

On 2015 Sep 15, Tom Kindlon commented:

(Contd.)

References:

1 Torjesen I. Tackling fears about exercise is important for ME treatment, analysis indicates. BMJ 2015;350:h227 http://www.bmj.com/content/350/bmj.h227

2 Chalder T, Goldsmith KA, White PD, Sharpe M, Pickles AR. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. Lancet Psychiatry 14 Jan 2015, doi:10.1016/S2215-0366(14)00069-8.

3 Burgess M, Chalder T. Manual for Participants. Cognitive behaviour therapy for CFS/ME.http://www.pacetrial.org/docs/cbt-participant-manual.pdf (accessed: January 17, 2015)

4 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME. Information for Participants http://www.pacetrial.org/docs/get-participant-manual.pdf (accessed: January 17, 2015)

5 Wechsler ME, Kelley JM, Boyd IO, Dutile S, Marigowda G, Kirsch I, Israel E, Kaptchuk TJ. Active albuterol or placebo, sham acupuncture, or no intervention in asthma. N Engl J Med. 2011;365(2):119-26.

6 Whiting P, Bagnall AM, Sowden AJ, Cornell JE, Mulrow CD, Ramírez G. Interventions for the treatment and management of chronic fatigue syndrome: a systematic review. JAMA. 2001 Sep 19;286(11):1360-8.

7 Burgess M, Chalder T. PACE manual for therapists. Cognitive behaviour therapy for CFS/ME.http://www.pacetrial.org/docs/cbt-therapist-manual.pdf (accessed: January 17, 2015)

8 White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, et al, for the PACE trial management group. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 2011;377:823-36.

9 McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD. Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. PLoS One. 2012;7(8):e40808. doi: 10.1371/journal.pone.0040808

10 Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg G. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychol Med. 2010 Aug;40(8):1281-7. doi: 10.1017/S0033291709992212. Epub 2010 Jan 5.

11 Heins MJ, Knoop H, Burk WJ, Bleijenberg G. The process of cognitive behaviour therapy for chronic fatigue syndrome: which changes in perpetuating cognitions and behaviour are related to a reduction in fatigue? J Psychosom Res. 2013 Sep;75(3):235-41. doi: 10.1016/j.jpsychores.2013.06.034. Epub 2013 Jul 19.

12 Friedberg F, Sohl S. Cognitive-behavior therapy in chronic fatigue syndrome: is improvement related to increased physical activity? J Clin Psychol. 2009 Apr;65(4):423-42. doi: 10.1002/jclp.20551.

13 Knoop H, Prins JB, Stulemeijer M, van der Meer JW, Bleijenberg G. The effect of cognitive behaviour therapy for chronic fatigue syndrome on self-reported cognitive impairments and neuropsychological test performance. Journal of Neurology and Neurosurgery Psychiatry. 2007 Apr;78(4):434-6.

14 Bavinton J, Darbishire L, White PD -on behalf of the PACE trial management group. Graded Exercise Therapy for CFS/ME (Therapist manual): http://www.pacetrial.org/docs/get-therapist-manual.pdf (accessed: January 17, 2015)

15 O'Dowd H, Gladwell P, Rogers CA, Hollinghurst S, Gregory A. Cognitive behavioural therapy in chronic fatigue syndrome: a randomised controlled trial of an outpatient group programme. Health Technology Assessment, 2006, 10, 37, 1-140.

16 Knoop H, Wiborg JF. What makes a difference in chronic fatigue syndrome? Lancet Psychiatry 13 Jan 2015 DOI: http://dx.doi.org/10.1016/S2215-0366(14)00145-X

17 Kindlon T. Reporting of Harms Associated with Graded Exercise Therapy and Cognitive Behavioural Therapy in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Bulletin of the IACFS/ME. 2011;19(2):59-111http://iacfsme.org/BULLETINFALL2011/Fall2011KindlonHarmsPaperABSTRACT/ta...

18 Lipkin DP, Scriven AJ, Crake T, Poole-Wilson PA. Six minute walking test for assessing exercise capacity in chronic heart failure. Br Med J (Clin Res Ed) 1986. 292:653–655. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1339640/pdf/bmjcred00224-001...

19 Marin JM, Carrizo SJ, Gascon M, Sanchez A, Gallego B, Celli BR. Inspiratory Capacity, Dynamic Hyperinflation, Breathlessness, and Exercise Performance during the 6-Minute-Walk Test in Chronic Obstructive Pulmonary Disease. Am. J. Respir. Crit. Care Med. 2001 63(6):1395-1399.http://171.66.122.149/content/163/6/1395.full

20 Goldman MD, Marrie RA, Cohen JA. Evaluation of the six-minute walk in multiple sclerosis subjects and healthy controls. Multiple Sclerosis 2008. 14(3):383-390. http://pocketknowledge.tc.columbia.edu/home.php/viewfile/download/65399/The six-minute walk test.pdf

21 Ross RM, Murthy JN, Wollak ID, Jackson AS. The six minute walk test accurately estimates mean peak oxygen uptake. BMC Pulm Med. 2010 May 26;10:31. PMID 20504351.http://www.biomedcentral.com/1471-2466/10/31

22 Camarri B, Eastwood PR, Cecins NM, Thompson PJ, Jenkins S. Six minute walk distance in healthy subjects aged 55–75 years. Respir Med. 2006. 100:658-65 http://www.resmedjournal.com/article/S0954-6111(05)00326-4/abstract

23 Troosters T, Gosselink R, Decramer M. Six minute walking distance in healthy elderly subjects. Eur Respir J. 1999. 14:270-4. http://www.ersj.org.uk/content/14/2/270.full.pdf

24 Rapport d'évaluation (2002-2004) portant sur l'exécution des conventions de rééducation entre le Comité de l'assurance soins de santé (institué auprès de l'Institut national d'assurance maladie invalidité) et les Centres de référence pour le Syndrome de fatigue chronique (SFC), Bruxelles, juillet 2006. (French language edition)

25 Evaluatierapport (2002-2004) met betrekking tot de uitvoering van de revalidatieovereenkomsten tussen het Comité van de verzekering voor geneeskundige verzorging (ingesteld bij het Rijksinstituut voor Ziekte- en invaliditeitsverzekering) en de Referentiecentra voor het Chronisch vermoeidheidssyndroom (CVS). 2006. Available online:https://drive.google.com/file/d/0BxnVj9ZqRgk0QTVsU2NNLWJSblU/edit (accessed: January 17, 2015) (Dutch language version)

26 Stordeur S, Thiry N, Eyssen M. Chronisch Vermoeidheidssyndroom: diagnose, behandeling en zorgorganisatie. Health Services Research (HSR). Brussel: Federaal Kenniscentrum voor de Gezondheidszorg (KCE); 2008. KCE reports 88A (D/2008/10.273/58)https://kce.fgov.be/sites/default/files/page_documents/d20081027358.pdf (accessed: January 17, 2015)

On 2015 Sep 15, Tom Kindlon commented:

Objective measures found a lack of improvement for CBT & GET in the PACE Trial: subjective improvements may simply represent response biases or placebo effects in this non-blinded trial

[Originally posted here: http://www.bmj.com/content/350/bmj.h227/rr-10]

This BMJ article and a flurry of articles in the lay media this week followed the publication in Lancet Psychiatry of an analysis of the mediators of change in the important PACE Trial, a chronic fatigue syndrome (CFS) trial which cost UK taxpayers £5 million[1,2]. What seems to have been lost in the coverage is that, although there were some modest improvements in the self-report measures, there was an almost complete absence of improvements in objectively measured outcomes for cognitive behavioural therapy (CBT) and graded exercise therapy (GET) compared to the control group (specialist medical care only (SMC)).

This is a non-blinded trial, where participants were told CBT and GET had previously been found to be effective in CFS and other conditions[3,4]: one way to look at the mediation results for subjective measures when there was a lack of objective improvements is that they may merely tell us how response biases and/or placebo effects are mediated[5].

The focus on subjective measures in some CFS studies was previously criticised in a systematic review published back in 2001 (long before the PACE Trial started)[6]. They suggested instead "a more objective measure of the effect of any intervention would be whether participants have increased their working hours, returned to work or school, or increased their physical activities."

The model presented for cognitive behaviour therapy (CBT) in the PACE Trial manuals posits that the impairments and symptoms are reversible with the therapy[3,7]. However, the latest paper shows that fitness, as measured by a step test, didn't improve following CBT[2]. An earlier PACE Trial publication reported that the addition of CBT to SMC did not result in an improvement in 6-minute walking test scores compared to SMC alone[8].

The PACE Trial was part funded by the UK Department of Work and Pensions, a rare move for them, presumably done due to an expectation that the therapies would improve measures of employment and levels of benefit receipt. However, again CBT brought about no improvement using objective measures, such as days of employment lost, levels of disability benefits received and levels of receipt of insurance payments[9].

These results are in line with earlier studies of CBT. For example, an analysis of three randomized controlled trials of CBT interventions for CFS found no improvement in objectively measured activity, despite participants reporting a reduction in (self-reported) fatigue and (sometimes) functional impairments[10]. Similar results were found in another uncontrolled trial where changes in objectively measured activity did not predict fatigue levels, and objectively measured activity on completion remained low compared to population norms[11]. An uncontrolled study found improvements in self-reported physical functioning and fatigue were reported despite a numerical decrease in (objectively measured) activity[12]. In another study, the level of self-reported cognitive impairment in CFS patients decreased significantly after CBT, however, cognition had not improved when it was measured objectively using neuropsychological test performance[13].

It is unsurprising that 15 sessions of CBT (and the associated homework exercises and management program) might alter how participants respond to self-report questionnaires. A PACE Trial manual itself says "the essence of CBT is helping the participant to change their interpretation of symptoms": this could lead to altered or biased fatigue scores, one of the two primary outcome measures[14]. Also, one of the aims of CBT (for CFS) has been said to be "increased confidence in exercise and physical activity"[15]. The possible responses for the other primary outcome measure, the SF-36 physical functioning subscale, are "yes, limited a lot", "yes, limited a little" and "no, not limited at all" to questions on a range of physical activities. Such responses could be easily be artificially altered following a therapy like CBT for CFS.

The results were not that different with the GET cohort in the PACE Trial. Again the manuals predicted that the impairments and symptoms are reversible using the intervention[4,15]. The model said there was no reason participants should not be able to get back to full functioning. Deconditioning was posited to be an important maintaining factor. However, GET did not result in an improvement in fitness, as measured by the step test. GET did result in a small improvement on the six minute walking test to a final distance of 379 metres, or 35 metres more than the SMC-only group[7]. However, as Knoop and Wiborg commented in an accompanying editorial in Lancet Psychiatry: "an increase in distance walked during a test situation without an increased fitness suggests that patients walk more because of a change in cognitive processes (eg, daring to do more or an increased self-efficacy with respect to activity), not because of a change in physiological capacity”[16]. The result remained very poor given that normative data would suggest a group of similar age and gender should walk an average of 644 or so metres[17]. The distance walked remained comparable to people with many serious conditions[18-21], and considerably worse than the distance walked by healthy elderly adults[22,23], despite the PACE trial cohort having a mean age of only 40[8]. Also, to be allowed entry into CFS research studies such as the PACE Trial one can not have a range of chronic illnesses so with genuine recovery one would expect results comparable to healthy people[8].

As with CBT, measures relating to employment showed no improvement following GET in days of work missed, which remained very high, nor a reduction in levels of benefits (financial support from the state) or payments from insurance companies[9].

These results are in line with an audit of Belgian rehabilitation centres for CFS offering CBT and GET[24-26]. Some improvements in subjective measures were found, but there was no improvement in the results of the exercise test and hours in employment actually decreased.

Probably the main contribution of the PACE Trial has been to add to a growing body of evidence that while CBT and GET for CFS have resulted in some changes on subjective measures, they haven't lead to improvements on objective measures.

Competing interests: I am a committee member of the Irish ME/CFS Association and perform various types of voluntary work for the Association.

(continues)

On 2015 Oct 15, Radboudumc Psycho-Oncology Journal Club commented:

This interesting commentary, which has important implications for the field of psycho-oncology, was discussed by our Journal Club members on 14th of October, 2015 and generated a lively discussion and the following comments and questions:

• 1) After reading your commentary, we concluded that it would be worthwhile to incorporate lifestyle variables in psychosocial intervention programmes on tertiary prevention, to further explore the relationship between psychological outcomes and health behaviours. Our group is based within an academic medical centre and there was debate within our group about the role of psycho-oncologists should take. Some members of our journal club believed that psycho-oncology as a discipline should focus more on collaboration with public health, health psychology and other disciplines working in the area of primary and secondary cancer prevention.. However, other members believed health behaviour change is the area of expertise of health psychologists not psycho-oncologists. Furthermore, in light of resource constraints, psycho-oncology professionals should focus on the core business of providing care to those affected by disease and ill-health in medical settings.<br>
• 2) We welcomed the proposal of The Expanded Model of Research in Psycho-Oncology and congratulate the authors for stimulating debate over the role of psycho-oncology. As for a suggestion, we would have liked it if the new conceptional model had an additional box with factors relevant to secondary prevention after cancer diagnosis. For example, psycho-oncologists are well placed to conduct activities that focus on the secondary prevention of psychological problems (e.g. depression, anxiety) or which might prevent a new disease recurrence following a diagnosis of cancer (e.g via lifestyle change programmes for cancer survivors). We also believed that psychological screening within oncology settings can be conceptualised as a secondary prevention activity.
• 3) We believe that psycho-oncology as a discipline, if it is to be involved with primary and secondary prevention at all, might focus on the three non-tumor specific health behavior changes with the biggest impact smoking, diet and exercise.
• 4) When it comes to medical psychologists working in psycho-oncology, essentially it comes down to the question whether a hospital is a “health valley” or a place in which we treat those affected by disease. Primary and secondary prevention implies the “health valley” model, which needs a much larger paradigm shift of medicine in general rather than in psycho-oncology alone. It will also require increased funding for hospitals to enable this to occur.<br>
• 5) We identified systemic barriers such as the organisational separation of hospitals and public health programmes in many jurisdictions. We concur with the authors that further investment is needed in the training of psycho-oncologists to ensure they learn how to apply their skills to preventive health and have more opportunities to gain work experience in preventive health care settings.

On 2015 Oct 02, Robert Groom commented:

Thank you to the authors for this tribute to an outstanding perfusionist that invested herself in improving her field. The commentary from her friends and contemporaries exemplifies the importance of our relationships with colleagues in professional societies and how through those relationships we are able to make the World a better place.

On 2015 Dec 06, Gwinyai Masukume commented:

Unacknowledged important limitation - the effective sample size of 229 was insufficient to detect a statistically significant association.

Dean and colleagues find that there was a deficit of 229 pregnancies among women who were in the first trimester when the mass shooting at Port Arthur occurred and that by sex there was no statistically significant differential loss. They estimate that for every 100 females, 107 males were lost.

Although Dean and colleagues did not find a statistically significant difference by sex, their sample size of 229 was insufficient to detect the difference given the small effect size Austad SN, 2015.

A rigorous study Orzack SH, 2015 demonstrated that normally, in the absence of exogenous stressors such as that caused by the Port Arthur calamity, more female embryos/fetuses are lost during the first trimester, approximately 100 females for every 97 males.

In conclusion, the results presented by Dean and colleagues show excess male loss during a period when more female loss is expected. Thus, the small sample size of 229 should have been mentioned as an important limitation of the paper and a formal power calculation was apt.

On 2015 Sep 17, Daniel Himmelstein commented:

Thanks Dr. Levine for your thoughtful response. As you mention, the practices I criticize in your discovery phase are not unique to your study. Your study caught my attention because of its remarkable finding that such a small sample yielded a highly predictive and robust classifier for such a complex phenotype. Hopefully, others will benefit from our discussion here.

Additionally, I agree that replication grants researchers the freedom to discover as they wish. Suboptimal model training should not cause "type I" replication error, if the replication dataset is independent.

However, a replication p-value alone is insufficient to identify the probability of a model being true. This probability depends on the plausibility of the model. Since I think that the odds are low that your study design could produce a true model, I require extraordinary evidence before accepting the proposed PRS model. I think your replication provides good evidence but not extraordinary.

Follow-up studies on different populations will be important for establishing extraordinary evidence. I think it would be helpful to specify which allele is minor for each PRS SNP: my impression is that minor alleles are sample not population specific.

On 2015 Sep 16, Morgan E Levine commented:

Daniel Himmelstein, thank you for your comments. I will try to address them to the best of my ability.

We acknowledge that the sample size is very small, which we mention in our limitations section of the paper. Because we are studying such a rare phenotype, there is not much that can be done about this. “Long-lived smokers” is a phenotype that has been the subject of a number of our papers, and that we think has strong genetic underpinnings. Despite the small sample size, we decided to go ahead and see if we could detect a signal, since there is evidence to suggest that the genetic influence may be larger for this phenotype compared to many others—something we discuss at length in our introduction section.

To the best of our knowledge the finding that highly connected genes contain more SNPs, has not been published in a peer-reviewed journal. Therefore, we had no way of knowing or evaluating the importance of this for our study. Similarly, we used commonly used networks and do acknowledge the limitations of these networks in our discussion section. The network you link to was not available at the time this manuscript was accepted.

We acknowledge the likelihood of over-fitting in our PRS, which is probably due to our sample size. This score did validate in two independent samples. Therefore, while it is likely not perfect, we feel that it may still capture some of the true underlying signal. We followed standard protocol for calculating our score (which we reference). In the literature there are many examples of scores that have been generated by linearly combining information from SNPs that are below a given p-value threshold in a GWAS. While, not all of these replicate, many do. Our study used very similar methods, but just introduced one additional SNP selection criteria—SNPs had to also be in genes that were part of an FI network. I don't think this last criteria would introduce additional bias that would cause a type I error in the replication analysis. However, we still recognize and mention some of the limitations of our PRS. We make no claim that the score is free from error/noise or that it should be used in a clinical setting. In fact, in the paper we suggest future methods that can be used to generate better scores.

We feel we have provided sufficient information for replication of our study. The minor alleles we used are consistent with those reported for CEU populations, which is information that is readily available. Thus, the only information we provide in Table S2 pertain to things specific to our study, that can't be found elsewhere. Lastly, the binning of ages is not 'bizarre' from a biogerontology and longevity research perspective. A number of leaders in the filed have hypothesized that the association between genes and lifespan is not linear (variants that influence survival to age 100+ are not the same as variants that influence survival to age 80+). Thus, using a linear model would not be appropriate in this case and instead we selected to look at survival by age group.

On 2015 Sep 15, Daniel Himmelstein commented:

I have several concerns with the discovery phase of this study. Specifically:

Underpowered: The sample size (90 cases) is insufficient for a complex phenotype where effect sizes are small. My own research, Himmelstein DS, 2015, does not consider GWAS with under 1000 samples because any findings are likely to be false (Sawcer S, 2008). The pathway-based prioritization attempts to alleviate the underpowered study design but suffers from the following two criticisms.

SNP abundance confounding: Genes were selected for the network analysis if they contained any SNPs with p < 5×10^-3. Therefore, genes containing more SNPs were more likely to get selected by chance. Since genes with more SNPs appear more frequently in curated pathway databases, the enrichment analysis is confounded. A permutation test that shuffles case-control status and recomputes SNP p-values would prevent SNP abundance confounding. However, this does not appear to be the permutation test that was performed.

Limited network relevance: The network analysis uses only curated pathway databases. These databases are heavily biased towards well-studied genes as well as being incomplete. In our recent network that includes more curated pathways than Reactome FI, only 9,511 genes are in any curated pathway. In other words, the majority of genes aren't curated to a single pathway and hence cannot contribute to this study's prioritization approach.

Overfitting: The polygenic risk score (PRS) was ruthlessly overfit. The PRS perfectly discriminated the 90 long-lived smokers from the younger smokers. The authors don't appear to appreciate that the performance is due to overfitting and write:

Results showed that the score completely accounted for group membership, with no overlap between the two groups.

Not only were scores significantly higher for the long-lived group, but scores also appeared to be more homogeneous.

Egregious overfitting is guaranteed by their PRS approach since 215 logistic regressions are fit, each with only 90 positives and without regularization or cross-validation. When a model is overfit on training data, its perfomance on novel data diminishes.

Unreplicable: Table S2 of the supplement does not specify which allele is minor for each SNP. Therefore, the PRS computation cannot be replicated by others.

Given these issues, I find it unlikely that the study found a reliable genotype of longevity. Rather, I suspect the successful validation resulted from confounding, p-value selection bias, or an implementation error.

Finally, the binning of continuous outcomes, primarily age, is bizarre. The binning serves only to reduce the study's power, while providing much room for unintended p-value selection bias.

Update 2015-09-15: I am not suggesting any misconduct, negligence, or intentional bad practices. Rather the methods are clearly described and the validation quite impressive and seemingly honest. I believe the study makes a valuable contribution by proposing a genotype of longevity, which future studies can confirm or deny.

Update 2015-09-19: I replaced "p-hacking" with "p-value selection bias". My intended meaning is the greater investigation and preferential publication granted to more significant findings.

On 2017 Oct 21, Anju Anand commented:

This article was discussed at our #rsjc. This captures the summary: http://www.thelancet.com/journals/lanres/article/PIIS2213-2600(15)00523-8/fulltext

On 2016 Jun 13, Stuart Phillips commented:

Please see a letter to the editor regarding the results from this paper: http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=10353765&fileId=S000711451600221X

On 2016 Apr 30, Stuart Phillips commented:

None

On 2016 Apr 30, Stuart Phillips commented:

None

On 2015 Sep 18, Alejandro Diaz commented:

Prepubertal gynecomastia is a very rare condition and there have been an unexpected number of cases reported in the Miami area. This case report highlighted three cases. However, we have seen many other children in our practices with prepubertal gynecomastia or thelarche who were similarly exposed to the lavender-containing cologne described in our report. Unfortunately, the evaluation and follow up for these children was not as in-depth as in the cases that we presented in the article, which included chemical analysis of the cologne product.

In Politano VT, 2013 article, uterotrophic assay, performed in immature female rats, was used to evaluate estrogenic activity in vivo. However, these rats were only treated with lavender oil for a period of 3 days, whereas the children we reported on were exposed to the lavender-containing cologne for several years. As described by Henley DV, 2007, the estrogenic activity of lavender is weak, which may explain why there was not uterotrophic effects on these rats.

You mentioned that the reason Henley found that lavender and tea tree oil activated the estrogen receptor was due to the use of polystyrene in their test system as opposed to the use of glass. Both Ohno K, 2003 and Fail PA, 1998 could not demonstrate estrogenic response from polystyrene. Therefore, this explanation is not substantiated. Moreover, there was an estrogenic response to the lavender and tea tree oils, but not to the control substance, despite having used the same test system containers in both conditions.

While it is true that there are additional components in many lavender preparations, we have not found prepubertal gynecomastia to be associated with other non-lavender-containing colognes or topical products. Thus, lavender itself is a logical and well-founded explanation for the physical findings in our patients.

Regarding tea tree oil, I previously evaluated a patient who was exposed over a period of several years to numerous products containing tea tree oil, including shampoos, toothpaste, detergents, home cleaning products, and melaleuca essential oil for minor cuts, burns, and other skin issues. He developed severe prepubertal gynecomastia that improved upon discontinuation of the exposure to these products. As I did in the cases of lavender exposure, I conducted a complete endocrine evaluation and his hormone levels were all within normal ranges. I did not publish this case at the request of his parents, who were in the tea tree oil industry. As clinicians, if we find patients with prepubertal gynecomastia who have been exposed for prolonged periods of time to substances known to activate the estrogen receptor, it provides substantial evidence of causality.

The possibility exists that there are contaminants that have estrogenic activity in these preparations. However, it is the responsibility of the industry that the products as sold are safe for human exposure.

It is, however, of utmost importance to publish these case reports to offer the balance you refer to, and to deepen the scientific knowledge base and protect consumers from unnecessary harm. Clinicians must be informed and use their clinical judgment to determine what is best for their individual patients.

Alejandro Diaz, M.D. and Marco Danon, M.D. Pediatric Endocrinologists

On 2015 Sep 14, Tony Larkman commented:

It is disappointing that the authors didn’t include or at least refer to the following two articles in this paper: 1. Politano VT et al (2013), Uterotrophic Assay of Percutaneous Lavender Oil in Immature Female Rats; International Journal of Toxicology; 32(2): 123-9 (http://www.ncbi.nlm.nih.gov/pubmed/23358464) 2. Carson CF et al. (2014), Lack of evidence that essential oils affect puberty; Reproductive Toxicology; 44:50-1 (http://www.ncbi.nlm.nih.gov/pubmed/24556344)

The first paper largely exonerates lavender oil through the ’gold standard’ uterotrophic assay while the second hypothesizes a mechanism that may be causal in incidences of both gynecomastia and premature thelarche as well as the ‘in vitro’ findings of Henley et al (2007) where they in fact used polystyrene in their test system and not glass. Based on these it is not impossible to conceive that other endocrine disruptors may have been inadvertently present in the material tested.

A further, as yet untested, hypothesis for these manifestations is the high incidence of adulteration in essential oils. The incidence of adulteration in TTO is remarkably high as reported in Wong YF et al. (2014) Enantiomeric distribution of selected terpenes for authenticity assessment of Australian Melaleuca alternifolia oil; Industrial Crops & Products; 67: 475-83 (http://ijt.sagepub.com/content/early/2013/01/24/1091581812472209) where more than 50% of 43 commercial samples tested failed to comply with the proposed chiral ratios. Of 15 commercially sourced samples in the European Union 73% of these showed significant differences in chiral abundances while TTO from both North America and Asia also displayed similar results where ≥50% of the tested samples did not match the expected results. In material of Chinese origin the incidence rises to 100%. An extraordinary range of compounds never found in pure TTO has been coincidentally detected so it has been further hypothesized that this, along with the likelihood that the material used is heavily oxidized, may be significant source of problems such as allergic contact dermatitis attributed to the use of TTO as well as conditions related to endocrine disruption.

It is disappointing for the tea tree oil industry as a whole that TTO continues to be mentioned as a potential endocrine disruptor without a more balanced view being presented when there are clear indications available in the literature that the original proposal by Henley et al in 2007 may be flawed. The fact that TTO was not present at all in any of the material tested is also disappointing as TTO can in no way be implicated yet its mention continues to promulgate the link and implicate TTO unfairly.

On 2018 Jan 07, Mark Milton commented:

In defense of the authors, their article was first published online on September 8, 2015 and appeared in the Issue published on April 1, 2016. The BIAL incident occurred in Jan 2016, i.e. after this article was published and hence there couldn't have been any discussion of the BIAL incident in this article.

On 2016 Jun 08, Christopher Southan commented:

Some discussion of the BIA-10-2474 fatality and AEs would have been pertinant, even though it was more recent than the cut-off date https://cdsouthan.blogspot.se/2016/01/the-unfortunate-case-of-bia-10-2474.html

On 2015 Dec 07, Sanjay Kumar commented:

We appreciate the interest in our work. While we are unable to comment on the internalization mechanisms of the SmartFlare reagents based on our data, our independent RT-PCR measurements (Fig. 3) are consistent with the SmartFlare data and confirm that matrix stiffness and ligand density regulate miR18a levels in this system.

On 2015 Nov 18, Raphael Levy commented:

This article uses the SmartFlare technology to detect miR18a. I would be most interested in the authors' opinion on how these probes escape endosomes to report on miR18a level. In our experience, they do not and fluorescence increase is most likely correlated with degradation in endosomes. It might be quite plausible that endocytosis "is non-linearly regulated by matrix stiffness and fibronectin density in glioma cells."

On 2015 Nov 12, Christopher Sampson commented:

It seems that personalised screening for breast cancer is a sheep in wolves' clothing (rather than the author's suggestion of the reverse).

The purpose of risk-based screening is to redirect resources to those at the greatest risk of disease; those in the greatest need. I have argued elsewhere that it is possible to prioritise screening in this way, in order to maximise the benefits of screening within a given budget.

Feig's commentary suggests that the terms "risk-based screening" and "personalized screening" are being misappropriated. That may well be true. However, the criticisms therein relate to very specific uses of these terms and to particular guidelines (I do not see these claims holding true more broadly). It does not then follow that 'risk-based screening' and 'personalisation' are wolves, and indeed the way they have been clothed does not appear desirable. They are sheep in wolves' clothing!

We should not -- as I hope Feig would agree -- allow the unsavoury dressing-up of these ambitious developments waylay further research into risk-based screening.

On 2017 Feb 18, Clive Bates commented:

Erroneous interpretations have been placed on these results and overly confident conclusions drawn from very small numbers of imperfectly characterised teenagers. The headline recommendations were based on the behaviour of six out of 16 baseline e-cigarette users in a sample of 694 adolescents deemed not to be susceptible to smoking. Large conclusions drawn from small numbers should always be a cause for caution, as discussed in this article about this study by Five Thirty-Eight:

Ignore The Headlines: We Don’t Know If E-Cigs Lead Kids To Real Cigs by Christie Ashwandan, 11 September 2015

One should expect the inclination to use e-cigarettes to be caused by the same things that cause an inclination to smoke - they are similar habits (the former much less risky) and it is quite likely that those who used e-cigarettes first would have become smokers first in the absence of e-cigarettes - a concept known as shared liability. A range of independent factors that create a common propensity to smoke or vape, such as parental smoking, rebellious nature, delinquency etc. explain the association between vaping and smoking incidence but without this relationship being causal.

The authors try to address this by characterising teenagers non-susceptible to smoking if they answer “definitely no” when asked the following: “If one of your friends offered you a cigarette, would you try it?” and “Do you think you will smoke a cigarette sometime in the next year?”. The study concentrates on this group.

This is not a foolproof way of characterising susceptibility to smoking, which in any case is not a binary construct but a probability distribution. Nor is susceptibility a permanent condition for any young person - for example, if a teenage girl starts seeing a new boyfriend who smokes that will materially changes her susceptibility to smoking. The fact that some were deemed unsusceptible to smoking but were already e-cigarette users is grounds for further unease - these would be more likely to be the teens where the crude characterisation failed.

It is a near-universal feature of tobacco control research that the study presented is a wholly inadequate basis for any policy recommendation drawn in the conclusion, and this study is no exception:

These findings support regulations to limit sales and decrease the appeal of e-cigarettes to adolescents and young adults.

The findings do not support this recommendation, not least because the paper is concerned exclusively with the behaviour of young people deemed not susceptible to smoking and, within that group, a tiny fraction who progressed from vaping to smoking. Even for this group (6 of 16) the authors cannot be sure this isn't a result of mischaracterisation and that they would not have smoked in the absence of e-cigarettes. The approach to characterising non-susceptibility is far too crude and the numbers involved far too small to draw any policy-relevant conclusions.

But this isn't the main limitation. Much more troubling is that the authors made this policy recommendation without considering the transitions among young people who are susceptible to smoking - i.e. those more likely to smoke, and also those much more likely to use e-cigarettes as well as or instead of smoking. This group is much more likely to benefit from using e-cigarettes as an alternative to smoking initiation, to quit smoking or cut down or as a later transition as they approach adult life.

There are already findings (Friedman AS, 2015, Pesko MF, 2016, Pesko MF, Currie JM, 2016) that regulation of the type proposed by the authors designed to reduce access to e-cigarettes by young people has had unintended consequences in the form of increased smoking - something that should not be a surprise given these products are substitutes. While one may debate these findings, the current study makes no assessment of such effects and does not even cover the population that would be harmed by them. With these limitations, it cannot underpin its own over-confident and sweeping policy recommendation.

On 2015 Sep 15, Aadil Inamdar commented:

The very aim of a systematic review is to provide a critical summary of the available evidence. The authors of a systematic review may draw conclusions based on the collected studies on a particular topic. It is equally important to present to their readers an unbiased quality assessment of the included studies in the review, which is an integral part of a systematic review process. This process is what differentiates a systematic review from the rest and a missing quality assessment of included studies section brings it as par with the common narrative or a literature review. The risk of bias and quality assessment of studies included in this review is missing. The onus is on the researchers to provide clear, simple conclusions of studies that has been meticulously checked and rechecked for their validity. Otherwise, on the contrary, it only helps to pollute the current evidence.

It is important to note that;

• Systematic reviews are one of the highest levels of information in the hierarchy of evidence LINK
• Steps involved in a systematic review have been laid down by the Cochrane collaboration STEPS
• A major section involves assessment of quality and risk of bias, of the included studies
• Assessment not only helps in ascertaining validity of a particular study, but also gives a true estimate (over/under) of the intervention (or exposure) Assessment of quality and risk of bias

On 2015 Nov 14, Arnaud Chiolero MD PhD commented:

An enlightening review revealing the issues of universal lipid screening in children. Similar issues with the screening of other CVD risk factors in children (see www.publichealthreviews.net/content/36/1/9)

On 2015 Sep 10, David Keller commented:

Did droxidopa increase dyskinesias, blood pressure, heart rate & adrenergic side-effects?

In general, medications which improve rigidity and bradykinesia and other "off symptoms" in Parkinson disease (PD) patients also tend to worsen dyskinesias. If droxidopa was able to improve PD "off" symptoms without worsening dyskinesias, then this drug is a breakthrough. The effect of droxidopa on dyskinesias is a crucial outcome and should be reported in the abstract. Without this information, the study cannot be evaluated properly.

Droxidopa is a metabolic precursor of norepinephrine, so it is expected to result in adrenergic effects, such as increasing blood pressure, heart rate, insomnia, heart arrhythmias, constipation, etc. The presence or absence of such side-effects is of interest to clinicians and patients, and should also be reported in the abstract.

Addendum

On September 16, 2015, I received a very informative email from Shifu Zhao, MD, in which he stated that "adrenergic side-effects were not significantly increased comparing with placebo group or baseline data at droxidopa dose 600mg/day" including "blood pressure, insomnia, palpitation, ECG" [1].

In addition, he assured me that add-on droxidopa therapy improved tremor and alternating motion of the hands in patients with moderate-to-severe Parkinson's disease, without worsening dyskinesias. [1]

I thank Dr. Zhao for supplying this information to me, and I hereby relay it to other Parkinson disease patients who read scientific abstracts without access to the underlying journal articles.

Reference:

1: Zhao S, Personal Communication by email, Received 9/16/2015 in reply to an earlier email inquiry.

On 2017 Mar 13, Atanas G. Atanasov commented:

Excellent overview on the topic, I have featured this publication at: http://healthandscienceportal.blogspot.com/2017/03/the-stomach-in-health-and-disease.html

On 2015 Oct 26, Gustav van Niekerk commented:

It would be hard to over emphasise the intense selective pressure that pathogens place on host populations (as predicted by the Red Queen hypothesis). Our genome are littered with pseudonized genes formerly associated with immune-related functions. These ‘genomic fossils’ representing discarded immunological innovations that became redundant as our pathogens evolve strategies to defeat them, testifying to an ancient conflict that have been ranging on between host and pathogens. Sequencing genomes have repeatedly demonstrated that genes with immunological functions tend to be under selective pressure and tend to be highly polymorphic (Mother Nature is ‘diversifying’ the ‘portfolio’ of immunological strategies: a pathogens can overcome many immunological strategies, but not all of them simultaneously. This observed immunological heterogeneity in population is how a species ‘hedge it’s bet’ against pathogens). Host and pathogen/parasites occasionally demonstrate "mirror-image phylogenies", demonstrating the close coevolution as pathogens keep in step with host development. Collectively, such observations suggest that pathogens exerts immense evolutionary pressure and one of the biggest drivers for evolutionary novelty.

Consequently, we suggest a less exiting narrative in which an AIS evolved initially in response to pathogen stress. Similar to your work Corcos D, 2015, we also noted that AIS evolved in an aquatic environment. Sediment and marine environments typically have a higher pathogen burden (see references in < PMID: 25698354 >), thus suggesting a higher level of pathogen stress. However, as you point out, there are scepticism about whether the AIS represent a true immunological innovation, as invertebrates do not seem any worse-off than vertebrates in suffering from infections. In this regard, we would like to point out that the AIS could have been an initial innovation that was subsequently overcome by fast evolving pathogens. That is, the AIS could have evolved in response to pathogen stress, providing an initial benefit to vertebrates, but was subsequently overcome by pathogens. And now, regardless whether the AIS is currently an ‘immunological innovation’, we are stuck with it (for better or worse). Finally, it is not obvious that the AIS do not repeat a lasting innovation. After all, “[v]ertebrates are the dominant group of animals on the Earth, given their abundance, large body sizes and presence at the top of both aquatic and terrestrial foodwebs” Wiens JJ, 2015. How did vertebrates manage to survive (and indeed flourish) in a world initially dominated by invertebrates? An AIS may have some additional benefits that have helped vertebrates invade into niches previously occupied by invertebrates. How AIS could provide a benefit that is (a) only applicable to vertebrates and (b) does not involve an enhancement of immune potency is currently an open question.

(PS we unfortunately do not have institutional access to the article from Burnet FM. We will shortly be sending a letter to the editor, commenting on your Corcos D, 2015 interesting article)

On 2015 Oct 20, Daniel Corcos commented:

This paper is interesting for examining the relationship between the adaptive immune system and the vascular system. However it does not explain such a complexity with no obvious advantage. For instance, immunodeficiency in zebrafish does not result in a dramatic change in viability. This can be interpreted in the words of Hedrick as an indication that the AIS is "not so superior"(1) or in the words of Burnet that it is "related to something other than defence against pathogenic microorganisms."(2) I have proposed (3) that its origin was related to intraspecific predation (cannibalism).

1)Hedrick SM. Immune system: not so superior. Science 2009;325:1623–4.

2)Burnet FM. “Self-recognition” in colonial marine forms and flowering plants in relation to the evolution of immunity. Nature 971;232:230–5.

3)Corcos D. Food-Nonfood Discrimination in Ancestral Vertebrates: Gamete Cannibalism and the Origin of the Adaptive Immune System. Scand J Immunol. 2015;82(5):409-17.

On 2017 Jul 04, Randi Pechacek commented:

Alex Alexiev wrote a blog post on microBEnet, mentioning this paper, arguing the importance of equipment cleanliness for food quality.

On 2016 Jan 20, thomas samaras commented:

A very interesting study. However, the conclusion that overweight may not be harmful at older ages is not consistent with what we know about the dangers of increasing BMI and various health parameters. Lamon-Fava showed many years ago that virtually all biological parameters get worse with increasing BMI in a linear trend; e.g., BP, fibrinogen, Apo B, Apo A, and HDL. In addition, the New England Centenarian Study found that almost all men who reached 100 years of age were lean. The Okinawan centenarians are also very lean. We also know that pre-western people with low chronic disease throughout their lives also tend to be quite lean in old age; e.g., people of Kitava (near Papua New Guinea).

Maier, Van Heemst and Westendorp found that shorter 85 and 90 years were more likely to reach advanced ages. These shorter, longer living individuals also had longer telomeres than taller people as found in the Guzzardi, et al. study. Perhaps confounding occurred by a number of lean people who were developing health problems without current symptoms, smokers, or poorly nourished individuals. This confounding may explain the results in regard to the positive correlation of increased % of fat mass and cholesterol between the start and end of the study.

On 2018 Jan 19, Natalie Clairoux commented:

Free version of this article available at http://hdl.handle.net/1866/19740

On 2016 Feb 10, Anders von Heijne commented:

In a recent case with a very large testicular epidermoid cyst we found an ADC-value of 0.544. IT seems that testicular epidermoid cysts have distinctly lower ADC-values than their intracranial counterparts.

On 2018 Jan 19, Natalie Clairoux commented:

Free version of this article available at http://hdl.handle.net/1866/19736

On 2015 Dec 08, S. Celeste Morley commented:

Thank you very much for your interest in and comment upon our work. The safety and efficacy of PCV in preventing and reducing the incidence of invasive pneumococcal disease is unquestioned and unequivocally supported by all literature. The mechanisms by which PCV protects are multifactorial. PCV generates an immune response that protects against invasive disease, if not colonization. We did not address incidence of IPD in this study. PCV also results in decreased carriage prevalence of the disease-causing serotypes covered by the vaccine, and thus shifts serotype prevalence without necessarily altering overall carriage prevalence. Our study simply reported overall carriage prevalence and the antibiotic susceptibility profiles of carriage isolates; we did not report serotypes. Our results finding that overall likelihood of colonization with any pneumococcal serotype is not affected by PCV is in line with other larger studies (e.g. Zuccotti et al Vaccine. 2014 Jan 23;32(5):527-34. Zuccotti G, 2014) Thus, our study is not in conflict with any of the studies referenced above, which clearly show a decrease in colonization with vaccine-covered serotypes.

On 2015 Dec 07, Manoochehr Karami commented:

Manoochehr Karami, PhD, Research Center for Health Sciences and Department of Epidemiology, School of Public Health, Hamadan University of Medical Sciences, Hamadan, Iran.

In an interesting study published recently, Julie Y. Zhou et al.(1) highlighted the prevalence of nasopharyngeal pneumococcal colonization among children in the greater St. Louis hospital. Authors have stated "pneumococcal conjugate vaccine (PCV) did not alter prevalence" of nasopharyngeal carriage. World Health Organization indicated that after PCV introduction, both targeted and non-targeted vaccination population were affected by direct and indirect effects of PCV immunization(2). Moreover, published studies (3-7) supported the changes of epidemiological profile of Streptococcus pneumonia related diseases transmission and nasopharyngeal carriage even among those individuals who were not immunized against Streptococcus pneumonia. Accordingly, it seems Zhou JY et al. interpretations based on their findings is in question and might be affected because of potential selection bias while enrolled participants. Rational for such selection bias is the catchment area of St. Louis hospital and potential differences between study participants and non-participants. Although they have excluded some patients, however this strategy does not guarantee the representativeness of their own work. Generally speaking, better explanation for Zhou et al findings is selection bias. In conclusion, lack of generalizability of this study findings should be considered by policy makers and interested readers.   References: 1. Zhou JY, Isaacson-Schmid M, Utterson EC, et al. Prevalence of nasopharyngeal pneumococcal colonization in children and antimicrobial susceptibility profiles of carriage isolates. International Journal of Infectious Diseases;39:50-52. 2. World Health Organization. Measuring impact of Streptococcus pneumoniae and Haemophilus influenzae type b conjugate vaccination. WHO Press, Geneva, Switzerland, 2012. 3. World Health Organization.Pneumococcal vaccines WHO position paper – 2012.Wkly Epidemiol Rec. 2012;87:129-244. . 4. Lehmann D, Willis J. The changing epidemiology of invasive pneumococcal disease in aboriginal and non-aboriginal western Australians from 1997 through 2007 and emergence of nonvaccine serotypes. Clinical Infectious Diseases. 2010, 50(11):1477–1486. 5. Pilishvili T, Lexau C. Sustained reductions in invasive pneumococcal disease in the era of conjugate vaccine. The Journal of Infectious Diseases, 2010, 201(1):32–41. 6. Davis S, Deloria-Knoll M, Kassa H, O’Brien K. Impact of pneumococcal conjugate vaccines on nasopharyngeal carriage and invasive disease among unvaccinated people: Review of evidence on indirect effects. Vaccine.2014;32:133-145. 7. Karami M, Alikhani MY. Serotype Replacement and Nasopharyngeal Carriage Due to the Introduction of New Pneumococcal Conjugate Vaccine to National Routine Immunization. Jundishapur Journal of Microbiology 2015;8.

On 2015 Sep 10, George McNamara commented:

Over at PubPeer (where this will be mirrored to) there were questions as to why I consider the image quality of figures 1 and 2 are so bad. See the Ma et al 2013 paper - also published in PNAS. http://www.pnas.org/content/110/52/21048.figures-only

Was not that hard two years ago to publish high quality images. Nor was it that hard for Deng et al to publish good quality figure 3 or supplemental files.

On 2015 Sep 09, George McNamara commented:

The good news of this paper: open access. The bad news: figures 1 and 2 are among the worst light microscopy image quality I have seen in a long time. I don't know if PNAS completely dropped the ball on image quality (fig 3 does look ok - also the authors do get to approve or fix the page proofs), or something to do with the authors. When I was a graduate student, I thought the difference between a cell biologist and a biochemist was that a cell biologist (or their lab) had microscopes and knew how to use them, and biochemists did not and did not care. Alternative hypothesis: PNAS figures are managed by biochemists. Considering the authors are down the hall from Eric Betzig, who won a prize of high resolution microscopy (and has since published two Science papers on more high res microscopy), they could have acquired and published better images. Their centromere, telomere repeats, MUC1 and MUC4 tandem repeat targets were all previously published (and cited by them) with somewhat better image quality.

On 2015 Oct 01, Badrul Arefin commented:

Dear readers,

We found two mistakes either as a missing vertical axis title or a typo in the following figures in this published article. In figure 7A, vertical axis title is missing when figure re-arrangement is performed in a revised version. Here it should be "Eclosure (%)". In figure 8I, N-NAME in chart area should be replaced with "L-NAME". Please excuse us for these mistakes. However, both of these are correctly-written/correct in the respective figure legends and elsewhere in the article.

Sincerely,

Badrul Arefin, On behalf of all authors.

On 2015 Sep 01, Friedrich Thinnes commented:

From my point of view the dust-raising effect of antibody 6E10 fuels the idea that plasmalemma-integrated VDAC-1 works as a receptor of amyloid Aß.

It has been shown that docking of amyloid Aß via GxxxG motif interaction to cell membrane-standing VDAC-1 opens the channel, and this way induces neuronal cell deaths accumulating over time.

Whenever critical brain regions and all redundant structures are affected Alzheimer´s Dementia appears.

For details see Friedrich P. Thinnes, Biochimica et Biophysica Acta 1848 (2015) 1410–1416 and www.futhin.de

On 2015 Sep 11, Chanin Nantasenamat commented:

Thanks Christopher for the reminder. Actually, we also wanted to make the dataset publicly available. Thus, please find the dataset used in this study (in Excel file format) from the following link:

http://dx.doi.org/10.6084/m9.figshare.1539580

On 2015 Sep 05, Christopher Southan commented:

This is a usefuly detailed study but direct availability of the 2,937 structures should have been a condition of publication. I note they were selected from BindingDB but also filtered post-download (and will change via updates). The authors should thus please surface at least the SMILES on figshare or other open acess option (Sep 14th - good 2 c the file :)

On 2015 Oct 23, Miguel Lopez-Lazaro commented:

This article provides a model to explain that virtually every cancer cell within a tumor often contains the same core set of genetic alterations, with heterogeneity confined to mutations that emerge late during tumor growth.

In my opinion, there is a simpler explanation. Recent evidence strongly suggests that cancer arises from normal stem cells. If cancer arises from normal stem cells, all the mutations occurring in these cells before becoming malignant (cancer stem cells, CSCs) will be found in all their progeny, that is, in all the tumor cancer cells. Some tumor cells may lack some of these mutations if they lose during cell division the chromosomes or pieces of chromosomes that bear these DNA alterations. The mutations arising during the self-renewal of CSCs will be found only in the tumor populations derived from these malignant stem cells. In addition to self-renewing, CSCs generate progenitor cancer cells, which divide and produce the bulk of cancer cells within a tumor. The mutations found in few tumor cancer cells probably occur during the division of these progenitor cells. In some cases, the tumor cancer cells may arise from more than one normal stem cell. In these cases, not all the cancer cells within a tumor will share the same core set of genetic alterations (1). Normal and malignant stem cells are defined by their self-renewal capacity and differentiation potential, and have a natural ability to migrate.

(1). Lopez-Lazaro M. Selective amino acid restriction therapy (SAART): a non-pharmacological strategy against all types of cancer cells. DOI: 10.18632/oncoscience.258

On 2015 Aug 28, Tom Kindlon commented:

Minimal change on only objective outcome measure:

On the only objective measure, the activity monitor, at 52 weeks (compared to baseline): the MRT group had increased by 5.8% and CBT group had increased by 6.6%.

There was no control group. One would expect a no therapy CFS group would likely on average increase their activity a little also in 52 weeks esp. in the early years of the illness and/or in the first year or two after diagnosis.

On 2015 Oct 03, Prashant Sharma, MD, DM commented:

The affiliation for Drs. Varma N, Sachdeva and Sharma should read as "Department of Hematology" and not "Hepatopathology".

On 2015 Sep 08, Alexandra Alexiev commented:

This article was written about in a blog post at: http://microbe.net/2015/09/08/antimicrobial-resistance-countermeasures-in-veal-farming/

MicroBEnet is a microbiology of the built environment blog funded by the Sloan foundation to communicate about scientific research in the field.

On 2015 Nov 26, Lydia Maniatis commented:

We can get a sense of the intellectual poverty of this study simply by reading the "Conclusions" section:

"Our experiments show that luminance edges play a central role in White's illusion. The illusion seems to be predominantly caused by the luminance edge between the test patch and its background bar, while the edge contrast to neighboring bars is largely ignored. The effect of contour adaptation on White's illusion could not be replicated by spatial filtering models, which adds further evidence against the adequacy of such models as a mechanistic explanation of White's illusion in particular, and lightness perception in general. Our results highlight the importance of further investigating the question of how surface lightness is computed from edge contrast. "

There is no content here because:

1) There is no alternative to the idea that "luminance edges play a central role in White's illusion." Both sides of the display are the same, other than the (perceived) location of the targets as lying on white or on black. Note that there is NO REFERENCE to the authors' orientation claims.

2) The inadequacy of a priori inadequate models that weren't even properly tested (assuming they could be) is not an argument for anything.

3). The statement that "Our results highlight the importance of further investigating the question of how surface lightness is computed from edge contrast" is meaningless. If there are questions to be answered, this study did not address them nor make them seem any more interesting.

The ad hotness of the authors' orientation claims is not only refuted by other displays, such as the Benary cross, but can also be refuted by versions of White's illusion in which the edges of the targets are curved so as to collectively be consistent with, e.g. an interrupted circle. Try it. Such a display, for me, makes more evident the fundamentally bistable character of the lighter-looking group of targets. They can either appear to be part of an opaque surface that lies behind the white stripes, on an amodally-completed black background, or they can appear to form part of a transparency that passes over the black and white stripes. The transparency aspect of White's illusion, which has been noted before, and is noticed by naive observers, is, of course, never touched on in this paper.

On 2015 Nov 08, Lydia Maniatis commented:

This article should probably not be read from the beginning, but from the end – from the last section of the discussion, titled “Difficulties with matching tasks.” The major theoretical problems discussed earlier seem moot once you appreciate that problems with method render the data virtually worthless..

The stimulus effects were not robust but, rather, highly ambiguous and liable to produce theoretically relevant effects that the authors did not bother to consider. This led to highly variable responses that they could not interpret. The authors try and put the blame for their data problems on the lightness matching technique itself, but the fault lies in them: If a task (or a stimulus) is not fit for purpose, it is the fault of those who chose it when it fails to deliver.

The problem is, in fact, the stimuli, which, again, were highly unstable and often did not produce the percepts that the authors predicted and which needed to arise reliably in order to allow them to validate their predictions. Four of ten observers, we are told, did not even perceive White's illusion! These observers were “following some strategy, but [our data] does not allow us to understand what exactly they are doing.”

In general, there was “large variability across and within observers....With simple stimuli of the kind used here, the perceived lightness of different image parts can change over time....In that case, the mean across trials may not be a good indicator of subjective experience [i.e. perception].” So the authors do not really know what their subjects are perceiving.

It seems, further, that the “simple stimuli” produced perceptual phenomena that the authors were or should have been aware are possible – either through the literature or simply on the basis of looking at their stimuli (the least a perception scientist should be doing when planning an experiment is to notice obvious effects of their stimuli) - but did not take into consideration: “Ekroll and Faul (2013) observed an additional transparent layer for similar stimuli...” This might “explain the inconsistent results...” So the “simple stimuli” could produce complex percepts - precepts that, for these authors simply amounts to noise.

In addition: “...observers sometimes selected match lightnesses that were outside the luminance range spanned by the grating. This is surprising....” They really have no idea what their data mean.

The authors' stunning conclusion: “This is just one further example that lightness matching is not such an easy and straightforward task as it might appear on the surface.” They don't seem to realize that no method is secure if your stimuli are ambiguous and unstable and you have not inspected them and factored the potential effects into your theory and method.

Unfortunately, lesson seems not to have been learned, as revealed by the final discussion section of Betz, Shapley, Wichmann and Maertens, 2015, which similarly describes devastating methodological problems as an afterthought. Why are reviewers setting such an obviously low bar for publication?

On 2015 Aug 30, Lydia Maniatis commented:

The authors of this study construct an untenable, ad hoc account of a myopic observation.

The observation is that the contrast effect in White's illusion “is largely determined by edge contrast across the edge orthogonal to the grating, whereas the parallel edge has little or no influence.”

This is a correct literal description of White's illusion. If we had no knowledge of any other contrast illusions, we might overgeneralize from this and conclude that, in general, orthogonal edges produce contrast effects while parallel edges do not. But we do know more. We know that contrast effects are not tied to edge orientation in this way. Consequently we would not reasonably attempt to construct a neural model of contrast based on a principle of “Orthogonal edges produce contrast, parallel edges don't,” since such an ad hoc model, shrink-wrapped to fit White's illusion, would instantly be challenged and falsified by any number of other contrast effects. However, this is precisely what Betz et al propose. (Not to mention that, being “low-level,” the authors' proposed mechanism would also have to be compatible with every other “low-level” as well as every “high-level” effect, e.g. transparency effects, since information from the early parts of the visual system is the basis for the high-level effects.)

In addition, the authors tailor their account to a very weak version of White's illusion (as the authors show, its effect is comparable to the classic slc demo), composed of single target bars rather than columns of aligned bars. The latter produce a much larger lightness differences as well as transparency effects (in the case of the lighter-seeming bars). Does the proposed model work for this classic version as well? Does it work for the classic simultaneous contrast illusion? Does it work for round targets on square backgrounds? Does it explain the Benary cross? If not, then how does such an account contribute to theoretical or practical understanding of the phenomenon (lightness contrast) under consideration? What are the chances that the visual system possesses a mechanism especially for producing a version of White's illusion?

As far as their experimental observations, the authors take a little too much credit for themselves when they say that their experiments have shown that “not all luminance borders that enclose a surface are treated equally in White's illusion.” This is an unnecessarily narrow framing of a many-decades-known, fundamental fact of lightness perception.

If there is one safe statement we can make about simultaneous contrast, it is that it is closely correlated with the appearance of a figure-ground relationship. Specifically, the surface that appears as figure lightens against a darker (apparent) background, and darkens against a lighter one. Mere (apparent) adjacency does not produce contrast. So when the authors claim to have shown that “the luminance step between the test patch and the grating bar on which it is [apparently!] placed is the critical condition for perceiving White's illusion” they a. Have not shown anything new and b. seem naive to the theoretical implications and generality of their own casual description (“on which it is placed”).

The implications are that, given the robustness of the figure-ground/simultaneous contrast link, the challenge of predicting simultaneous contrast reduces to the challenge of predicting figure-ground relations. The latter, in contrast, is not reducible to structure-blind theories involving inhibition/filtering based on luminance ratios. And it is certainly not reducible to an orthogonal/parallel edge dichotomy.

On 2015 Oct 07, Bart Verkuil commented:

An Associate Editor of The Journal of The Norwegian Medical Association made us aware of an error in the Introduction of this paper: we mention that a meta-analysis of the longitudinal relation between workplace bullying and mental health, published in The Journal of The Norwegian Medical Association (Tidsskrift for Den norske legeforening) in 2014, is only available in the Norwegian language. This is incorrect, as it is available in English: http://www.tidsskriftet.no/article/3213422

On 2016 Aug 23, Ben Goldacre commented:

This trial has the wrong trial registry ID associated with it on PubMed: both in the XML on PubMed, and in the originating journal article. The ID given is NCT013655104. We believe the correct ID, which we have found by hand searching, is NCT01365104.

This comment is being posted as part of the OpenTrials.net project^[1] , an open database threading together all publicly accessible documents and data on each trial, globally. In the course of creating the database, and matching documents and data sources about trials from different locations, we have identified various anomalies in datasets such as PubMed, and in published papers. Alongside documenting the prevalence of problems, we are also attempting to correct these errors and anomalies wherever possible, by feeding back to the originators. We have corrected this data in the OpenTrials.net database; we hope that this trial’s text and metadata can also be corrected at source, in PubMed and in the accompanying paper.

Many thanks,

Jessica Fleminger, Ben Goldacre*

[1] Goldacre, B., Gray, J., 2016. OpenTrials: towards a collaborative open database of all available information on all clinical trials. Trials 17. doi:10.1186/s13063-016-1290-8 PMID: 27056367

* Dr Ben Goldacre BA MA MSc MBBS MRCPsych<br> Senior Clinical Research Fellow<br> ben.goldacre@phc.ox.ac.uk<br> www.ebmDataLab.net<br> Centre for Evidence Based Medicine<br> Department of Primary Care Health Sciences<br> University of Oxford<br> Radcliffe Observatory Quarter<br> Woodstock Road<br> Oxford OX2 6GG

On 2016 Apr 22, Falk Leichsenring commented:

For a response to Hofmann, see http://ebmh.bmj.com/cgi/content/full/eb-2016-102372?ijkey=giCBgiE7JTlTg&keytype=ref&siteid=bmjjournals

On 2016 Mar 25, Stefan Hofmann commented:

For a more detailed critique and response, see: http://www.ncbi.nlm.nih.gov/pubmed/27009055

On 2016 Feb 27, Falk Leichsenring commented:

We absolutely agree: strong conclusions require strong evidence

Falk Leichsenring, Patrick Luyten, Mark J. Hilsenroth, Allan Abbass, Jacques P. Barber, John R. Keefe, Frank Leweke, Sven Rabung, Christiane Steinert

Hofmann [1] asked us to provide evidence - here it is.

In our first response [2] to Hofmann et al. [3], we showed that their commentary on the quality of studies of PDT is not consistent with empirical evidence from quality research performed in adversarial collaboration between PDT and CBT researchers, which found no significant differences in quality between studies of PDT and CBT [4, 5]. Furthermore, we noted that Gerber et al. did not find significant correlations between methodological quality and outcome in studies of PDT [5]. These results (i.e., evidence) are inconsistent with Hofmann et al.’s conclusions [3], regardless of whether Hofmann et al.’s ratings [3] are methodologically sound.

In addition, we emphasized that Hofmann et al. failed to demonstrate any evidence that the quality of the 64 RCTs leads to results in favor of PDT. In a similar way Bhar and Beck suggested that the lack of difference in outcome between CBT and PDT found by Leichsenring, Rabung, and Leibing [6] was due to poor treatment integrity [7]. However, using Bhar and Beck’s own integrity ratings, their assertion was not corroborated by empirical data [8]. It is of note that these meta-analyses [6, 8] included researchers from both CBT (E.L.) and PDT (e.g. F.L.).

In our first response, we also observed that the authors failed to provide basic data on interrater reliability, raters’ training, the rating procedures, attempts to address allegiance effects, or blinding of raters [2]. The authors also did not include researchers of both approaches among the raters, as done by Gerber et al. and Thoma et al. [4, 5]. In addition, we noted that Hofmann et al. based their conclusions of poor methodological quality on "unclear" designations of quality [2]. Most authors would have attempted to contact the original authors of a study before asserting that procedural information was unclear and making strong conclusions about study quality.

Hofmann et al. [3] drew strong conclusions about the quality of our review using extreme terms such as "invalidating the authors´ results" and "making the findings meaningless" using nonstandard procedures of questionable quality. For strong conclusions, strong evidence is required. Yet, Hofmann et al. failed to provide it. For a commentary aiming to address study quality, it is puzzling to apply procedures of such poor quality.

We are raising these issues again since Dr. Hofmann did not address them in his response [1]. Instead of doing so, Dr. Hofmann stated [1]: ”As their only defense, the authors argue that CBT is also poorly supported." In this way, he is simply ignoring the evidence we provided and the methodological shortcomings of his commentary we had pointed out [2]. Further, we did not question that CBT is an efficacious treatment. We just pointed out that the available evidence shows that the quality of CBT studies is no better than that of PDT studies [4, 5].

We also did not intend to attack Dr. Hofmann on a personal level, but rather intended to provide evidence that he repeatedly applied double standards when judging studies of CBT as compared to those of PDT [2, 9, p. 49-51]. We respectfully asked that if he chooses to write about PDT (e.g., comment on a meta-analysis, conduct a meta-analysis, or conduct a study involving PDT), that he considers involving a psychodynamic researcher in the process. This invitation still stands.

Dr. Hofmann emphasized that CBT is widely disseminated in the UK. This is true, but PDT is recommended by treatment guidelines and implemented in the National Health Service in the UK as well. This is also true in other countries. In Germany, for instance, PDT is as frequently used as CBT [10]. The Scientific Board for Psychotherapy (Wissenschaftlicher Beirat Psychotherapie; WBP) is the paramount body in Germany for assessing the scientific status of psychotherapeutic interventions. For this purpose, standardized and transparent criteria are used. Based on a careful evaluation by the WBP, both CBT and PDT were acknowledged as scientific and efficacious forms of psychotherapy (www.wbpsychotherapie.de). It is noteworthy that the WBP is composed of researchers from diverse psychotherapeutic orientations (e.g. CBT, PDT, and systemic therapy). The studies of PDT were evaluated by CBT researchers, and vice versa. The conclusions by a balanced expert institution such as the WBP are incompatible with those by Hofmann et al. [3].

We all should be happy that a variety of psychotherapeutic treatments exist that are beneficial to patients. Future research should address the question of which patients benefit most from which treatments, and why.

Declaring the evidence of a whole treatment approach as "meaningless" is not supported by the preponderance of evidence, and is counter-productive to this goal.

References

1 Hofmann SG: Show us the data! Pubmed Commons Feb 17 2016 2 Leichsenring F, Luyten P, Hilsenroth MJ, Abbass A, Barber JP, Keefe JR, Leweke F, Rabung S, Steinert C: Once again: Double standards in psychotherapy research - response to hofmann et al. PubMed Commons 2016 3 Hofmann SG, Eser N, Andreoli G: Comment from pubmed commons. . January 23rd, 12:28am UTC 2016 2016 4 Thoma NC, McKay D, Gerber AJ, Milrod BL, Edwards AR, Kocsis JH: A quality-based review of randomized controlled trials of cognitive-behavioral therapy for depression: An assessment and metaregression. Am J Psychiatry 2012;169 5 Gerber AJ, Kocsis JH, Milrod BL, Roose SP, Barber JP, Thase ME, Perkins P, Leon AC: A quality-based review of randomized controlled trials of psychodynamic psychotherapy. Am J Psychiatry 2011;168:19-28. 6 Leichsenring F, Rabung S, Leibing E: The efficacy of short-term psychodynamic psychotherapy in specific psychiatric disorders: A meta-analysis. Arch Gen Psychiatry 2004;61:1208-1216. 7 Bhar SS, Beck AT: Treatment integrity of studies that compare short-term psychodynamic psychotherapy with cognitive-behavior therapy Clin Psychol-Sci Pr 2009;16:370-378. 8 Leichsenring F, Salzer S, Hilsenroth M, Leibing E, Leweke F, Rabung S: Treatment integrity: An unresolved issue in psychotherapy research. Curr Psych Rev 2011;7 313-321. 9 Leichsenring F, Rabung S: Double standards in psychotherapy research. Psychother Psychosom 2011;80:48-51. 10 Albani C, Blaser G, Geyer M, Schmutzer G, Brähler E: Outpatient psychotherapy in germany from the patient perspective: Part 1: Health care situation Psychotherapeut 2010;55:503-514.

On 2016 Feb 17, Stefan Hofmann commented:

Show us the data! The authors wrote in the abstract that psychodynamic therapy is as efficacious as treatments established in efficacy. Strong conclusions require strong evidence. Dr. Leichsenring and colleagues were unable to provide the reader with such evidence. Our earlier commentary of the article by Leichsenring and colleagues reported data suggesting that the majority of studies included in their review were of low quality. It is the responsibility of Leichsenring and colleagues to provide the reader with evidence that their conclusions are justified. Instead of providing the reader with such evidence, the authors chose to attack me on a personal level. Whether or not Dr. Leichsenring and colleagues believe that I am adequate to serve as a reviewer or editor of scientific journals or grants and collaborate with others is completely unrelated to the weaknesses of their study. As their only defense, the authors argue that CBT is also poorly supported. The authors are incorrect. The supporting evidence of CBT is overwhelmingly large. Our own review identified 269 meta-analytic studies of CBT. We observed that the quality of studies that entered some of these meta-analyses were not uniformly high. However, some of them were of high quality (e.g., Hofmann & Smits, 2008). Because CBT has such a solid empirical basis, many countries, including the UK, disseminate CBT on a large-scale basis, e.g., http://www.iapt.nhs.uk/. It should be noted that this dissemination is not limited to CBT but also includes other empirically supported treatments. Polemics and personal attacks on my scientific integrity are not the real problem. The biggest concern in my view is that these disputes distract from the real issue. They confuse our patients and policy makers, inhibit scientific progress, and inflict harm by withholding effective treatments. References 1. Hofmann, S. G., Asnaani, A., Vonk, J. J., Sawyer, A. T., & Fang, A. (2012). The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive Therapy and Research, 36, 427-440. doi 10.1007/s10608-012-9476-1 2. Hofmann, S. G. & Smits, J. A. J. (2008). Cognitive-behavioral therapy for adult anxiety disorders: A meta-analysis of randomized placebo-controlled trials. Journal of Clinical Psychiatry, 69, 621-632. doi: 10.4088/JCP.v69n0415

On 2016 Jan 29, Falk Leichsenring commented:

Once again: Double standards in psychotherapy research - response to Hofmann et al.

Falk Leichsenring, Patrick Luyten, Mark J. Hilsenroth, Allan Abbass, Jacques P. Barber, John R. Keefe, Frank Leweke, Sven Rabung, Christiane Steinert

Referring to a recent review on psychodynamic therapy (PDT) [1], Hofmann et al. [2] criticize the quality of studies included in this review. The authors conclude that the "poor quality" of studies of PDT "invalidates" the results of this review making them "meaningless" [2]. The comment by Hofmann et al. [2] deserves some response.

• The conclusions drawn by Hofmann et al. [2] are inconsistent with present research. As shown by an independent research group including proponents of both CBT and PDT working respectfully together, the quality of studies of PDT does not differ significantly from that of studies of CBT which fell into the lower range of adequate quality [3, p. 22, 4]. Most of the studies included by Leichsenring et al. [1] were also included in this comparison [3, 4]. However, Hofmann has not described the respective CBT studies as “meaningless”.

Furthermore, the comment by Hofmann et al. [2] suffers from several shortcomings.

• The authors are incorrect when referring to our publication as a meta-analysis. In fact it was a systematic review [1]. This is of note since possible shortcomings in individual studies would not invalidate the review as a whole.

• The authors draw a highly generalizing conclusion without any differentiation, for example, by disorders or degree of risk.

• For their ratings, Hofmann et al. [2] did not report basic data on the number and training of raters, on blinding, or interrater-reliability. For the best minimization of bias, raters of both approaches would have been included as done by Gerber et al. and Thoma et al. [3, 4]. Thus, the quality of the procedures applied by Hofmann et al. [2] themselves is questionable.

• The conclusions by Hofmann et al. [2] are based mostly on “unclear” designations, not clear flaws. In fact, an “unclear” risk of bias indicates that the design feature could be both worse or better than described in the article. What is most concerning is that the authors did not make any effort to resolve the “unclear” assignments by carefully reading the papers or contacting their authors. Many assignments are obviously clear from the studies [e.g. 5, 6].

• In addition, even if there are flaws, Hofmann has not shown that these particular flaws lead to results in favor of PDT (rather than e.g., greater error in effect estimates overall). Several meta-analyses did neither find significant correlations between ratings of methodological quality and outcome [4, 7] nor between treatment integrity (assessed by prominent CBT researchers, e.g. Aaron T. Beck) and differences in outcome between CBT and PDT [8].

• If the "poor quality" of PDT studies "invalidates" [2] the results reported by Leichsenring et al. [1] making them "meaningless", this would equally apply to meta-analyses carried out by CBT researchers who included several of these same studies. Tolin [9], for example, included 10 studies also included by Leichsenring et al. [1]. Hofmann, however, has never critically commented on these meta-analyses - which are interpreted as supporting the efficacy of CBT - thus, again applying a double standard.

• Hofmann was repeatedly shown to apply double standards when judging studies of CBT vs. PDT [10, p. 49-51].

(a) In a previous meta-analysis, for example, Hofmann [11, p. 180] claimed that the quality of studies included by him was "considerably better" than that of studies of PDT, e.g. in the meta-analysis by Leichsenring and Rabung [12]. Hofmann et al. [11] reported a mean Jadad score of 1.23, whereas the mean Jadad score in the meta-analysis by Leichsenring et al.[12] was 1.96 (0 = poor; 5 = excellent).

(b) Hofmann [11]criticized the meta-analysis by Leichsenring et al. [12] for including heterogeneous studies. However, between-effect hterogeneity was in the low to medium range [10, 12]. In his own meta-analysis, Hofmann [11] did not even test for heterogeneity before combining data of randomized controlled and observational studies [10, 11].

• Thus, from a scientific perspective, it is questionable whether a strong proponent of CBT who has publicly demonstrated that he is an opponent of PDT [e.g. 13] is able to provide unbiased conclusions about PDT.

Given the author’s very negative publicly expressed opinions about PDT, and the way he conducted this critique of the Leichsenring et al. review [1] it appears that biases can lead to a lack of even-handedness in regard to the evaluation of psychodynamic studies. Thus, we respectfully would ask that if he chooses to write about PDT(e.g., comment on a meta-analysis, conduct a meta-analysis, or conduct a study involving PDT), that he involve a psychodynamic researcher in the process (i.e., implement a version of adversarial collaboration [14], and also that he recuse himself from being involved as an editor, or reviewer, in regard to research involving PDT.

We would welcome the collaboration of CBT researchers in researching psychotherapy and synthesizing the results from trials. We have done so several times [e.g. 15, 16].

Given the present crisis of replicability of research [17], biased and tendentious statements as those by Hofmann bear the risk of damaging all psychotherapy research equally in the eyes of the public.

References

1 Leichsenring F et al.: Psychodynamic therapy meets evidence-based medicine: a systematic review using updated criteria. Lancet Psychiatry 2015;2:648-660. 2 Hofmann SG et al.: Comment from PubMed Commons. January 23rd, 12:28am UTC 2016 3 Thoma NC et al.: A quality-based review of randomized controlled trials of cognitive-behavioral therapy for depression: an assessment and metaregression. Am J Psychiatry 2012;169 4 Gerber AJ et al.: A quality-based review of randomized controlled trials of psychodynamic psychotherapy. Am J Psychiatry 2011;168:19-28. 5 Barber J et al.: Short-term dynamic psychotherapy versus pharmacotherapy for major depressive disorder: a randomized, placebo-controlled trial. J Clin Psychiatry 2012;73:66-73. 6 Crits-Christoph P et al.: Psychosocial treatments for cocaine dependence: National Institute on Drug Abuse Collaborative Cocaine Treatment Study. Arch Gen Psychiatry 1999;56:493-502. 7 Keefe JR et al.: A meta-analytic review of psychodynamic therapies for anxiety disorders. Clin Psychol Rev 2014;34:309-323. 8 Leichsenring F et al.: Treatment integrity: An unresolved issue in psychotherapy research. Curr Psych Rev 2011;7 313-321. 9 Tolin DF: Is cognitive-behavioral therapy more effective than other therapies? A meta-analytic review. Clin Psychol Rev 2010;30:710-720. 10 Leichsenring F et al.: Double standards in psychotherapy research. Psychother Psychosom 2011;80:48-51. 11 Hofmann SG et al.: The effect of mindfulness-based therapy on anxiety and depression: a meta-analytic review J Consult Clin Psychol 2010; 78 169-183. 12 Leichsenring F et al.: Effectiveness of long-term psychodynamic psychotherapy: a meta-analysis. JAMA 2008;300:1551-1565. 13 Rief W et al.: [Saving psychoanalysis. At any cost?]. Nervenarzt 2009;80:593-597. 14 Mellers B. et al.: Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychol Sc 2001;12:269-275. 15 Leichsenring F et al.: The efficacy of short-term psychodynamic psychotherapy in specific psychiatric disorders: a meta-analysis. Arch Gen Psychiatry 2004;61:1208-1216. 16 Leichsenring F et al.: Psychodynamic therapy and CBT in social anxiety disorder: a multicenter randomized controlled trial. Am J Psychiatry 2013;170:759-767. 17 Open Science Collaboration: PSYCHOLOGY. Estimating the reproducibility of psychological science. Science 2015;349:aac4716.

On 2015 Oct 05, Lydia Maniatis commented:

It's been a month since I posted my comments on this article and on the original Blakeslee and McCourt article. A sharp increase in visits to the target articles post-comment seems to indicate that the comments are being read, but no author or reader has put in their two cents, either here or on PubPeer. I think that my argument against the "brightness-is-perceived-luminance" idea is sound and straightforward. Am I wrong?

On 2015 Sep 06, Lydia Maniatis commented:

I fully agree with the latter part of this commentary but disagree with the author's concession that, in lightness experiments, "when illumination appeared homogeneous, lightness and brightness judgments were identical. There is nothing new here. It is well known."

Brightness is currently being described, including by Gilchrist, as the perceptual correlate of luminance. But there is no perceptual correlate of luminance, even under (apparently) homogeneous illumination, and this can be proved as follows:

We ask an observer to report on the lightness of a set of surfaces which don't produce the impression of shadows or transparency. Then, in a second session, we present the same set of surfaces under a different level of illumination. The lightness reports for the surfaces will stay essentially the same, even though their luminances may have changed substantially. So to say that people are making "brightness" judgments in either the first or the second or in any case doesn't seem reasonable.

On 2018 Jan 24, Doug Berger commented:

Excellent article from a brave young researcher willing to put her career in the line of fire of criticism of CBT and other psychotherapy adherents, not to mention employment opportunities where psychotherapy research has been a mainstay for years and is a cash-cow. The only caveat I would add is that the translation of basic science research into the ability og CBT to overcome the inability to single-(patient blind), or double-(patient and therapist blind), as well as the inability to have blind placebo, in a psychotherapy clinical trial is a serious doubt. Many psychiatric study drugs over the years have had considerable pre-clinical basic science data, even phase I or phase II clinical data, only to fail miserably in phase III trials when adequate numbers of subjects were studied in tightly blinded conditions with blind placebo control. Basic science research only lends itself to decide if it is worth while to test a drug in large populations clinically, it doesn’t say itself if a drug will work, especially in psychiatric conditions where endpoints are subjective and random error high.

The whole premise of CBT, that negative or distorted cognitions are the cause of a psychiatric condition is the only instance in all of medicine and psychiatry where a symptom of an illness is also construed to be the cause. Taking a symptom and making it into a cause is a way to spin the need of a therapy aimed at a cause that actually the result, and in psychiatric conditions with subjective endpoint hope and expectation effects are an easy way to garner some symptom reduction that can easily be called a "response" in a clinical trial. Reading the paper, I wonder if the journal asked the authors to propose ways to work around the blinding problem. The work-around is heuristic but not of practical value once a clinical trial gets going-and as the authors rightly note (and as I have noted in other papers below), there is really no way to do blind/blind placebo psychotherapy clinical trial.

Other papers on this topic: https://www.ncbi.nlm.nih.gov/pubmed/26870318 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4863672/ https://www.japanpsychiatrist.com/Abstracts/CBT_Escape.html

D. Berger, U.S. Board Certified Psychiatrist

On 2015 Aug 25, Ranjith Ramasamy commented:

https://www.youtube.com/watch?v=qAaSG4LDSE4&feature=youtu.be - Video from Urology times summarizing the study findings

On 2015 Aug 23, Kelly Drew commented:

Hello to all! Interesting paper, but don't dismiss A1AR. Have you seen these papers?

1: Muzzi M, Coppi E, Pugliese AM, Chiarugi A. Anticonvulsant effect of AMP by direct activation of adenosine A1 receptor. Exp Neurol. 2013 Dec;250:189-93. doi: 10.1016/j.expneurol.2013.09.010. Epub 2013 Sep 19. PubMed PMID: 24056265.

2: Muzzi M, Blasi F, Masi A, Coppi E, Traini C, Felici R, Pittelli M, Cavone L, Pugliese AM, Moroni F, Chiarugi A. Neurological basis of AMP-dependent thermoregulation and its relevance to central and peripheral hyperthermia. J Cereb Blood Flow Metab. 2013 Feb;33(2):183-90. doi: 10.1038/jcbfm.2012.157. Epub 2012 Oct 24. PubMed PMID: 23093068; PubMed Central PMCID: PMC3564191.

3: Muzzi M, Blasi F, Chiarugi A. AMP-dependent hypothermia affords protection from ischemic brain injury. J Cereb Blood Flow Metab. 2013 Feb;33(2):171-4. doi: 10.1038/jcbfm.2012.181. Epub 2012 Dec 5. PubMed PMID: 23211965; PubMed Central PMCID: PMC3564206.

On 2015 Aug 26, Jim Woodgett commented:

I sound like a broken record, but it is quite remarkable that yet another paper presumes an inhibitor is specific for GSK-3beta. In this case (AR-A014418) is equally as effective at inhibiting GSK-3alpha, as is every other small molecule inhibitor of GSK-3, including lithium. This is a problem because the authors fail to assess the contribution of the second isoform which is being equivalently blocked in the experiments. There are numerous papers that describe the common and distinctive functions of these protein kinases.

On 2016 Feb 08, Salzman Lab Journal Club commented:

This paper presents a striking view of the intron lariat spliceosome at unprecedented resolution and uses pombe as a strategy for enriching relatively homogenous structural complexes. As our lab is interested in circular RNA, we were curious about how this structure may be used to model minimal circularized exon lengths. We look forward to more structures in the future that give a more detailed look at the exon-exon junction!

On 2016 Nov 03, David Juurlink commented:

Post-publication note: Additional analysis of absolute risk of opioid-related death among patients receiving more than 200 mg morphine (or equivalent) per day reported here http://bit.ly/2fVaQYL

On 2015 Aug 22, Arnaud Chiolero MD PhD commented:

An enlightening commentary on the necessity of revisiting DCIS treatment. It also raises question about screening - the major driver of DCIS identification.

On 2015 Nov 26, Tero Kivelä commented:

This paper is commented in http://www.ncbi.nlm.nih.gov/pubmed/26288835

On 2015 Nov 26, Tero Kivelä commented:

This is a commentary on http://www.ncbi.nlm.nih.gov/pubmed/26288849