The Role of Evidence in Social Science

There is no law school in the U.S. (nor, I would venture, in the U.K.) where the curriculum fails to address the importance of social science to law practice. Regrettably, social scientists do not always understand that the lessons can also run in the other direction, especially when it comes to the marshaling of facts. This became apparent to me when I read a review of my recent book, Interrogating Ethnography, on the British website Social Science Space.  The reviewer, a consulting sociologist named Robert Dingwall, panned the book, which of course happens to every author.  But he also missed the entire point of the subtitle – Why Evidence Matters – which I fear may tell us something more troubling about the practice of sociology.

Dingwall, who is co-editor of the SAGE Handbook of Research Management, is highly skeptical of documentary fact-checking, which he considers a “second-best method.” Yes, documentation has its limitations. But so do human informants, witnesses, and interview subjects, whose statements, as lawyers well know, are predictably affected by poor memory, self-interest, skewed perspective, implicit biases, the desire to please, and sometimes outright prejudice or deception. Dingwall may believe that ethnographers can effectively weed through all that, but the best of them – for example, Matthew Desmond, Mary Pattillo, Terry Williams, Kathryn Edin, and, again, Mitchell Duneier – realize that documentary fact-checking is an essential tool for social scientists.  Interrogating Ethnography therefore illustrated various means of fact-checking ethnographies, with results that ranged from disproof to solid verification.

This brings us to the heart of Dingwall’s review, which is his defense of Alice Goffman’s On the Run. I had no plans to revisit these issues – having already said my piece – but Dingwall’s argument presents a good opportunity to elucidate the role of factual accuracy in social science.

To show that my methods of verification are unacceptable, Dingwall says, “We can illustrate this by reference to a section of Goffman’s account that Lubet makes great play with, namely the avoidance of emergency rooms by young black men with injuries or other medical conditions for fear of screening or arrest.” He then claims to refute my critique by positing that “Goffman could be reporting a genuine belief based on a mistaken inference” which he himself has “no problem in imagining.”

There is little doubt that Goffman’s informants really believed that they risked arrest in emergency rooms, which I acknowledged several times in Interrogating Ethnography.  The problem, however, is that Goffman claims to be reporting on more than mistaken perceptions.  She states that the police do in fact set up cordons at the emergency room door:

To round up enough young men to meet their informal quotas and satisfy their superiors, the police wait outside hospitals serving poor Black communities and run the IDs of the men walking inside. (55)

Goffman says this twice more in On the Run, both times as truth claims about what she calls a “standard practice,” and not as an account of mistaken inferences. I doubted the story and therefore set out to investigate it. Because Goffman has refused to identify the hospital where the events allegedly occurred, I contacted personnel from every hospital that could reasonably be described as serving the poor Black communities of Philadelphia. Everyone denied the existence of police gantlets at the door.

Dingwall brushes off my fact checking with a jaunty Britishism that usually implies mendacity: “Well, they would say that, wouldn’t they?” Not that my informants were necessarily lying, he quickly adds. Perhaps they were only “buffered from knowledge” of the true goings on.

“Lubet is simply uncritical about his own ‘evidence,’” Dingwall charges, ignoring the fact that my “evidence” (the scare quotes are his) also included a statement from Philadelphia’s former deputy mayor for public security – an African American who had earlier been a public defender for over twenty years – to whom I provided the relevant excerpts from On the Run.  “The passage about hospitals is not in any way a standard practice,” he said, “It is not a practice, period.” I also interviewed two other Philadelphia officials, a high-ranking police officer, and two long-time public defenders, all of whom refuted Goffman’s claim. In addition, I cited three highly regarded ethnographies of similar populations in the same Philadelphia neighborhoods – two of them by African Americans, one of which was conducted in hospital emergency rooms – none of which mentioned police cordons.  Finally, I cited five other journalists – writing in Philadelphia Magazine, New York Magazine, the Chronicle of Higher Educations, Slate, and The Atlantic – who examined Goffman’s hospital story and found no trace of it in reality.

The question must be asked: If this evidence is insufficient for Dingwall, what could possibly satisfy him?

It takes impressive intellectual gymnastics to conclude that the ethnographer is presumptively right and everyone else may be so “buffered from knowledge” that there is no point to investigating further. Dingwall actually suggests a possible police conspiracy to falsify arrest locations, thus keeping the hospital ID checkpoints a secret, to this day, from everyone except Goffman. (Tip: If your analysis involves a hidden conspiracy, there is probably something wrong with it.)

The purpose of an ethnographic trial – analogous, though not identical, to a court trial or the replication of a randomized controlled trial – is to seek out and evaluate potentially disconfirming evidence.  Sociologists, too, should apply a standard of proof more rigorous than “I can imagine it.”

It is revealing that Dingwall’s review never once mentions that my book’s subtitle is Why Evidence Matters.  Then again, he has rather unintentionally proven that it does.

Comments

[Law School Cynic]

"There is no law school in the U.S. (nor, I would venture, in the U.K.) where the curriculum fails to address the importance of social science to law practice."
You've apparently never been to Creighton.

[Deep State Special Legal Counsel]

I am not sure if the issue here is good social science method or factual evidence. It's more of a young, immature Donald Trump type of problem. We are talking about a young, immature, unseasoned reporter who got too emotionally involved with her "subjects." At worst its a lack of integrity, at best its inexperience. Good social science data collection nor plays to legalistic evidence gathering will cure Trump nor her.

[Steve L.]

Prof. Goffman (one assumes) has moved on to other subjects. The greater issue here is that a seasoned sociologist like Dingwall cannot see the problem with reporting mistaken beliefs as though they are established facts.

[Enrique Guerra-Pujol]

Has Dingwall refused to update his prior?