Do the Editors at BMJ Archives of Disease in Childhood Care about Conflicts of Interest?

Fate has compelled be to pay attention to medical studies and journal articles on Chronic Fatigue Syndrome (ME/CFS). I am usually most interested in an article's abstract, describing the results, as I am not easily able to closely follow the medical details. Sometimes, however, there are ethics issues regarding the studies themselves, which are squarely in my wheelhouse. So it was with a recent study, in the British journal Archives of Disease in Childhood, of the so-called “Lightning Process,” which seemed to validate the three-day training process – which uses “neuro-linguistic programming,” life coaching, and “emotional self-regulation” – as a potentially effective treatment for ME/CFS in children.

Biomedical research ethics are not the same as legal ethics, but there are areas of overlap, including conflicts of interest. A central rule in both fields should be that individuals cannot be relied upon to assess their own objectivity. Instead, we have external standards for judges, lawyers, and, one would have thought, medical researchers. Regrettably, the editors of Archives of Disease in Childhood have allowed the authors of the Lightning Process study to vouch for their own credibility, in circumstances where their credibility should have been reasonably in doubt.

 The invaluable David Tuller, of the University of California, Berkeley, reviewed the Lightning Process study and discovered several glaring problems. Most significantly, the investigators recruited more than half the participants before registering the study, which violated the policy of all major medical journals on only publishing trials that adhere to strict prospective registration procedures. Then the investigators swapped the primary and secondary outcomes following the early results, without acknowledging this change or the registration irregularities when publishing the study.

 As Tuller explained, 

 The original primary outcome in the feasibility trial—school attendance at six months—was relegated to the status of a secondary outcome. The subjective measure of self-reported physical function, which was a secondary measure for the feasibility trial, became the primary outcome for the full trial.

This is significant, because the study actually showed no improvement for school attendance at six months. Swapping primary and secondary outcomes, however, allowed the study to report effectiveness based on the positive but subjective results for self-reported physical function at six months. In other words, the authors changed the rules – and in fact, changed the very scoring system – in the middle of the game. Or in poker terms, they decided that a straight would beat a flush, but only after the cards started lining up. Nonetheless, the study received widespread media coverage for this claimed success.

Tuller reported this and other breaches in a series of reports on Virology Blog and letters to Archives of Disease in Childhood and BMJ, which publishes the journal. More than a year later, Archives has finally posted a lengthy “correction.” This correction acknowledges the deficiencies noted by Tuller, including the late trial registration, but accepts the “assurance from the authors that the change in primary outcome was not influenced by (positive) findings in the feasibility phase.”

 This is where I come in. Having learned that the investigators swapped outcomes -- without even mentioning it in their original paper – the BMJ editors should never have accepted their “assurance” that the change was not influenced by the authors’ own interest in achieving positive findings. That is the very definition of a conflict of interest. Even if the authors were sincere in believing they had not been influenced by the greater publishability of a positive result, that was not a judgment they should have been allowed to make. We have conflict of interest rules precisely because individuals cannot be relied upon – simply as a matter of cognition science – to recognize or evaluate their own biases. 

Here is what may have happened in the Lightning Process article. The investigators initially specified school attendance at six months as their primary outcome, with self-reported improvement on physical function as a secondary measure. Presumably it was possible to sense early outcome trends even without analyzing any data, since it was an unblinded trial relying on subjective reports. If the data from the early participants indicated that school attendance had not improved, it makes sense that the investigators would have changed the primary outcome to one that was more promising--which they subsequently did without mentioning the switch. 

Upon learning of this unacknowledged substitution, the BMJ editors relied on the authors’ own explanation that the changeover was not “influenced” by their self-interest in obtaining better results. In the world of professional ethics, we call this an “apparent conflict of interest,” of the sort that cannot be resolved by the assurance of an interested party, even if provided in full good faith. Readers familiar with judicial ethics will recall that recusal is mandatory in circumstances where "impartiality might reasonably be questioned."

 There is an old saying in law, that no person can be a judge in their own cause. We apply it pretty rigorously, and it is an embarrassment when we do not. The BMJ editors have evidently taken the opposite view, allowing authors to judge the impact of their own lapses. They ought to be embarrassed.

[Disclosure: Tuller also organized an open letter to the editor of Archives of Disease in Childhood, on which I was a signatory.]

Comments

[Sten Helmfrid]

In poker terms, Professor Crawley bet all her money on jack high in a game of Hold’em. When all the cards were on the table, she declared that they now were playing lowball and raked in all the chips in the pot. The BMJ Editor—who managed the poker game—declared that he accepted the assurance from Professor Crawley that she intended to switch to lowball already before she saw her own cards.

[Nancy Blake]

In her paper, Crawley acknowledges that many children did not want to enter the trial. Reports from others who have completed the Lightning Process (and been made much worse by doing so) indicate that the NLP processes for belief change and increasing motivation were used to get patients to deny that they were sick (told to say they were fine no matter how they felt) and to enact this denial by giving up aids and practices they had adopted to manage an illness characterised by muscle failure upon minimal exertion, and exercising.

The US IOM (now Academy of Medicine) report on ME/CFS
http://www.ncbi.nlm.nih.gov/books/NBK284910/
states that this is a chronic, disabling disease which of which the central characteristic is that ‘exertion of any kind - physical, cognitive or emotional - may adversely affect many organ systems...’. It is not psychogenic.

This confirms that encouragement to believe otherwise, and to behave otherwise, is a serious level of medical abuse. Additionally, the children were told that the treatment ‘wouldn’t work if they told anyone what it was’.

This adds psychological abuse to the physical abuse. The treatment includes being told to lie about how you feel. But you mustn’t tell anyone that it does. Now researchers are asking you to answer a questionnaire about how much the treatment has helped, and they are persistent (ringing parents repeatedly) in their demands for replies.

This fulfils the three conditions for what Gregory Bateson calls a ‘schizophrenogenic double bind’. The person is asked to do two opposite things at the same time (lie/tell the truth), cannot comment on their situation (mustn’t tell anyone what the treatment is), and cannot escape from the situation (researchers persistently demanding answers). Bateson believed that this could cause a psychotic break with reality. I believe that there has been one suicide.

http://www.positivehealth.com/article/cfs-me/me-cfs-nlp-and-the-lightning-process-in-the-looking-glass

Not only should the treatment be identified as criminal medical abuse, so should the trial itself.