Naomi Wolf's embarrassment on live BBC Radio has been widely reported, but the facts are even worse than explained in most accounts. In brief, Wolf's new book, Outrages: Sex, Censorship, and the Criminalization of Love, includes her assertion that dozens of British men were executed for homosexual acts in the nineteenth century, which she claims to have discovered searching records at the Old Bailey. In an interview with the BBC's Matthew Sweet, she said:
I found several dozen executions . . . . and this corrects a misapprehension that is in every website, that the last man was executed for sodomy in Britain in 1835.
Her evidence for the discovery, which she believed had been missed by all other scholars (including historians), was the notation "death recorded" next to conviction for the sodomy.
Sweet pointed out to her that "death recorded . . . doesn't mean that he was executed." Rather, it was a category created in 1823 "that allowed judges to abstain from pronouncing sentence of death on any capital convict whom they considered to be a fit subject for pardon." In fact, Sweet continued, "I don't think any of the executions you've identified here actually happened." (Sweet has a doctorate from Oxford and has written a book about the Victorian era.)
In other words, Wolf's major conclusion was based on a misunderstanding of British terminology. The full BBC interview is here; the crucial exchange is here; the story in New York Magazine is here.
All of that is bad enough, but here is why it was even worse:
Wolf's book is based on her doctoral dissertation at Oxford, which she completed in 2015 (she had also been a Rhodes Scholar). Thus, her advisor and the members of her dissertation committee must have seen the erroneous assumption and overlooked it, which does not speak well for the rigor of her program. Should they have known about the meaning of "death recorded"? Well, her advisor was Stefano-Maria Evangelista, whose field is "Nineteenth-century English literature, especially Aestheticism and Decadence." That should have given him at least passing familiarity with British penal law, which formally imposed the death penalty for a broad range of felonies. In practice, however, executions were frequently suspended or commuted, as judges were loath to see them carried out.
Oxford requires two other examiners — one internal and one external — who participate in the candidate's oral defense. I do not know who the examiners were for Wolf's dissertation, but it is unlikely that either was a historian. Even if unfamiliar with the term "death recorded," a legal historian would have recognized the discontinuity between formal sentencing and actual executions, and at least would have checked out the meaning of the notation.
And where did Matthew Sweet find the meaning of the "death recorded" notation? Well, it happens to be right on the Old Bailey's website. Wolf made her supposed discovery by "looking at the Old Bailey records and the crime tables," and yet she didn't take the obvious step of looking up the definitions on the website, which is the primary tool for researching criminal justice records in London. There is a searchable data base of 197,745 trials conducted 1674-1913, which Wolf presumably used in her work.
The New York Times reports:
A spokeswoman for Houghton Mifflin Harcourt, which is publishing “Outrage” in the United States, referred to the issue as an “unfortunate error” but said, “we believe the overall thesis of the book ‘Outrages’ still holds.”
She added that while the publisher “employs professional editors, copyeditors and proofreaders for each book project, we rely ultimately on authors for the integrity of their research and fact-checking.”
Everyone makes mistakes, and no book is without errors. In this case, however, the sweeping mistake undermines a central premise of the work itself, in which Wolf wrongly claimed to have made a discovery that eluded all previous researchers. It's too bad that no one — her publisher, her editor, her advisor — seems to have shown the manuscript to a legal historian. Peer review does have its virtues.
Oh no.
This case shows why understanding one's cognitive biases is important for critical thinking and an essential part of being a scholar. Wolf had a thesis. Once she found evidence supporting that thesis she stopped; her theory had been confirmed. She had no need to look for contradictory evidence. This is called the Confirmation bias: "The tendency to search for, interpret, focus on and remember information in a way that confirms one's preconceptions."
A scholar who was aware of cognitive biases would have said I have to make sure my cognitive biases are not misleading me. Such a scholar would look harder for contradictory evidence than confirming evidence.
The publisher's statement is just as bad. “We believe the overall thesis of the book ‘Outrages’ still holds.” This statement exhibits a cognitive bias of the worst kind. You say that a central premise of the work had been undermined (as have several other critics), but to the publisher the work is just fine. This is called the Semmelweis reflex: "The tendency to reject new evidence that contradicts a paradigm."
E. Scott Fruehwald, Understanding and Overcoming Cognitive Biases For Lawyers And Law Students: Becoming a Better Lawyer Through Cognitive Science (2018).
Great comment, Scott. Of course, the great problem with cognitive biases, also called cognition errors, is that they are invisible to us, even when we think we are adjusting for them. That is why corrective systems — such as peer review and fact checking — are so important and need to be routinized.
Perhaps needless to say at this point, I have encountered unfathomable resistance to routine fact checking in my work on ethnography.
On the Semmelweis reflex, however, check out Sherwin Nuland's "The Doctors' Plague," in which he argues that Semmelweis's introduction of antisepsis was actually accepted by the Viennese medical establishment.
Interesting, the Semmelweis story may be an urban legend. However, the principle remains valid.
Yes, cognitive biases are very relevant to your ethnography criticisms.
The Wikipedia entry says that Semmelweis's conclusions were rejected by the medical establishment and not adopted for some years, so I don't know if Nuland was right or wrong — but it is a very fascinating book, and also very short, so I recommend it.
The publisher is just trying to salvage the book. No need to look for cognitive biases to explain its actions.
"Well, her advisor was Stefano-Maria Evangelista, whose field is "Nineteenth-century English literature, especially Aestheticism and Decadence." That should have given him at least passing familiarity with British penal law, which formally imposed the death penalty for a broad range of felonies."
Seriously?
It seems to me that this is one of those times when a book's release should be postponed for the book to be corrected. Obviously, since the book has not been released, I have not seen it, so I have no way of knowing whether there is anything in it worth saving once the necessary corrections have been made. If, as has been asserted, the remaining content is still accurate and makes the same points, the thesis will be more unassailable if this error is eliminated.
Failure to correct this error gives those who would oppose the underlying politics of the book ammunition to attack its conclusions in a way that should make all scholars cringe. There is also a real danger into the future that those who read the book will assume that the content is accurate; information on the blogosphere to the contrary that is not in the book will fade, but the book won't.
This is also what happens when one choses to go the easy route and get a Ph.D. in politicized English rather than do the heavy lifting and study history or sociology. As noted, she should at least have put a couple of outside experts on her committee.