My occasional coauthors David Tuller (UC Berkeley) and Brian Hughes (National University of Ireland) have an important new article in the Journal of Health Psychology. They provide a devastating critique of an ME/CFS study recently published in the Journal of the Royal Society of Medicine. As Tuller and Hughes explain, the underlying study suffered from “serious methodological flaws and the highly misleading way in which its results had been presented,” and “key statistical statements in the paper’s Abstract were factually inaccurate.” But that is not why I am posting about it.
Most alarming, from an academic perspective, was the way in which the Royal Society of Medicine first accepted a deeply problematic paper, and then refused to publish a subsequent critique. Here is how David Tuller described it on the Virology Blog, noting that one of the coauthors of the dubious paper was Prof Simon Wessely, the immediate past president of the Royal Society:
Another question is why would the purportedly rigorous Journal of the Royal Society of Medicine publish a paper that included misleading figures in a study abstract and made unwarranted causal claims? What happened with the peer review process that such obvious failings were not noticed, or not revised if they were? Did the journal recognize that it would need to be especially rigorous when reviewing a paper co-authored by the sponsoring society’s immediate past president, and that any weaknesses exposed later on would reflect especially badly on all involved?
If the initial publication may have reflected some in-group favoritism, the Royal Society certainly reinforced that impression by its subsequent decision. Here is how Tuller and Hughes summarized the critique they submitted to the Journal of the Royal Society of Medicine:
Their research is undermined by several methodological limitations, including: (a) sampling ambiguity; (b) weak measurement; (c) survivor bias; (d) missing data and (e) lack of a control group. Unacknowledged sample attrition renders statements in the published Abstract misleading with regard to points of fact. That the paper was approved by peer reviewers and editors illustrates how non-rigorous editorial processes contribute to systematic publication bias.
Even so, as Hughes explained on The Science Bit:
Despite this, the Journal of the Royal Society of Medicine saw fit to reject our submission without providing any reason. Even though our manuscript met all the journal’s guidelines, the editor chose to not even allow it be reviewed for publication.
Fortunately, the peer reviewed Journal of Health Psychology accepted and has now published the Tuller & Hughes article, in which they identify an additional problem with the Royal Society’s peer review:
More notably, the fourth peer reviewer is a psychiatry trainee on the Maudsley Training Programme. This programme is operated in partnership with the Institute of Psychiatry, Psychology and Neuroscience at King’s College London and the South London and Maudsley NHS Foundation Trust. All five co-authors of the Adamson et al. paper list one or both of these institutions as their academic affiliation(s). Two of the co-authors (…) hold senior positions at the Institute of Psychiatry at King’s. Another co-author (…) serves as a College Tutor and Clinical Skills Programme Co-ordinator on the Maudsley Training Programme itself. Ordinarily, being affiliated to the same institution(s) as a manuscript co-author would disqualify a person from serving as peer reviewer. Being affiliated not only to the same institution, but to the same academic units as all five co-authors would certainly appear to present significant issues concerning potential conflicts of interest. That the peer reviewer in question is a trainee of the co-authors’ institutions further compounds the problem. In our view, to ask a student to serve as peer reviewer for a paper produced by senior professors of their own college places them in an impossible position, and is highly inappropriate.
In sum, the Tuller and Hughes experience exposes a major pitfall in academic publishing about ME/CFS, in which old school eminences are able to push weak and contradictory studies, while entrenched journals resist even considering contrary viewpoints.
Comments