On 2016 Aug 29, VINCENT RACANIELLO commented:
Below is the text of a letter sent to The Lancet earlier this year, asking for an independent review of the PACE trial. The Lancet invited us to submit this letter, then rejected it for no reason. To keep the text within the limits of PubMedCommons, only the first six names of those who signed the letter are shown.
Sir: The PACE trial reported that exercise and psychotherapy were effective treatments for chronic fatigue syndrome. Yet PACE was an unblinded study with subjective primary outcomes—a design already vulnerable to bias. As patients have long noted, PACE also suffered from additional methodological lapses likely to produce unreliable results. A recent investigation by David Tuller of the University of California, Berkeley, heightened public awareness of PACE’s flaws and sparked an open letter from us to The Lancet requesting an independent review.
Among PACE’s serious missteps:
*In a paradox, participants could—before treatment—already qualify as improved or “within the normal range” for fatigue and physical function, the primary outcomes. How? Because PACE’s “normal range” outcome scores actually allowed for worse health status than the eligibility scores required at entry to demonstrate serious disability. At entry, 13 percent of participants scored “within normal range” on one or both measures—details unreported in the paper. One PACE investigator described participants meeting these lax “normal ranges” as “back to normal”—a misleading statement highlighted in news coverage. An accompanying commentary, reviewed pre-publication by the PACE investigators, called this “a strict criterion for recovery.”
*PACE claimed success based solely on subjective outcomes. Participants in one arm improved slightly on a walking test but remained severely disabled. The investigators subsequently dismissed this test and other objective measures--which all failed to demonstrate success--as irrelevant or non-objective. They describe their findings as robust, but unblinded trials with subjective endpoints, even without PACE’s many other problems, can only yield low-quality evidence.
*PACE used one symptom to identify participants—six months of disabling, unexplained fatigue. A 2015 National Institutes of Health report recommended abandoning this broad definition because it generates heterogeneous samples that could “impair progress and cause harm.” PACE’s sub-group analyses of two alternate criteria are also difficult to interpret—an unknown number of patients might have met these criteria but been excluded by the initial definition and thus unobserved.
*PACE significantly revised its protocol strategy for assessing primary measures and “recovery” criteria but did not include sensitivity analyses, the accepted method of evaluating the impact and dispelling concerns about bias. Whether or not investigators made revisions before reviewing outcome data is irrelevant: Outcome trends in unblinded trials are often apparent to trial leadership early on, even if assessors do not know treatment assignment. The investigators have declined to provide the protocol results and rebuffed data requests as “vexatious.”
*A newsletter for participants--published while a third were still undergoing assessment-- included glowing testimonials from earlier participants about excellent outcomes but none reporting poor outcomes, potentially biasing subjective ratings toward the positive. The same newsletter reported that new U.K. treatment guidelines, “based on the best available evidence,” recommended the two PACE interventions favored by the investigators. Adaptive Pacing Therapy, an untested intervention developed specifically for PACE, was not mentioned.
*Despite investigators’ explicit protocol promise to inform participants of “any possible conflicts of interest,” consent forms did not mention their ME/CFS-related work for disability insurers. It is irrelevant that they disclosed these conflicts in The Lancet and that insurers played no role in PACE. Given the omission, the signed consents are of questionable legitimacy.
The investigators’ previous responses to such concerns have been unsatisfactory. We firmly believe the trial data should undergo a fully independent review.
Vincent R. Racaniello, PhD Professor of Microbiology and Immunology Columbia University
Ronald W. Davis, PhD Professor of Biochemistry and Genetics Stanford University
Jonathan C.W. Edwards, MD Emeritus Professor of Medicine University College London
Leonard A. Jason, PhD Professor of Psychology DePaul University
Bruce Levin, PhD Professor of Biostatistics Columbia University
Arthur L. Reingold, MD Professor of Epidemiology University of California, Berkeley
This comment, imported by Hypothesis from PubMed Commons, is licensed under CC BY.