A remarkable new “sting” of the "diet research-media complex" was just revealed. It tells us little we didn’t already know and has potentially caused a fair amount of damage, spread across millions of people. It does, however, offer an opportunity to explore the importance of prospective group review of non-consensual human subjects research—and the limits of IRBs applying the Common Rule in serving that function in contexts like this.
Journalist John Bohannon, two German reporters, a doctor and a statistician recruited 16 German subjects through Facebook into a three-week randomized controlled trial of diet and weight loss. One-third were told to follow a low-carb diet, one-third were told to cut carbs but add 1.5 ounces of dark chocolate (about 230 calories) per day, and one-third served as control subjects and were told to make no changes to their current diet. They were all given questionnaires and blood tests in advance to ensure they didn't have diabetes, eating disorders, or other conditions that would make the study dangerous for them, and these tests were repeated after the study. They were each paid 150 Euros (~$163) for their trouble.
But it turns out that Bohannon, the good doctor (who had written a book about dietary pseudoscience), and their colleagues were not at all interested in studying diet. Instead, they wanted to show how easy it is for bad science to be published and reported by the media. The design of the diet trial was deliberately poor. It involved only a handful of subjects, had a poor balance of age and of men and women, and so on. But, through the magic of p-hacking, they managed several statistically significant results: eating chocolate accelerates weight loss and leads to healthier cholesterol levels and increased well-being.
They wrote up the results, submitted the manuscript to twenty predatory journals known to publish basically anything for a price, picked one of the “multiple” ones that accepted it, and wrote a check. (The journal that published the paper now says it did so “by mistake” and has removed it from its website, but you can read it here.) Then they cooked up a press release, sent it out to numerous journalists and media outlets, and waited. Several (of various quality, focus, and circulation) took the bait and passed on to their readers the good news that adding chocolate to a "strict" low-carb diet would "accelerate weight loss." As Bohannon brags:
It made the front page of Bild, Europe’s largest daily newspaper, just beneath their update about the Germanwings crash. From there, it ricocheted around the internet and beyond, making news in more than 20 countries and half a dozen languages. It was discussed on television news shows. It appeared in glossy print, most recently in the June issue of Shape magazine (“Why You Must Eat Chocolate Daily”, page 128).
Then, yesterday, Bohannon exposed his own “con” in io9, an online science and technology magazine, under a headline that blared: I Fooled Millions Into Thinking Chocolate Helps Weight Loss.
Was the sting ethical?