Blind spots

A great post over at The Situationist blog about a new book from Mahzarin Banaji and Anthony Greenwald called Blind Spot: The Hidden Biases of Good People which is an attempt to bring some of their work on the Implicit Associations Test (IAT) to a broader, general audience. This work is interesting and provocative and, as noted in the blog, often controversial. But I suspect the book will be very interesting to those legal academics who have either written about the IAT or issues of implicit bias or cognitive limitations generally, as well as to practitioners who may be coming up against this sort of evidence in court or who, conversely, would like to use it in court.

Update: TFL blogger Michelle Meyer adds this review of the book from the WSJ blog,  Book Review: Blind Spot. After looking at the review I think I'd have to say it goes beyond simply offering a caution about importing the insights (if any) of the IAT, but is fairly hostile to this research.  It offers several of what seem like valid criticisms and a few that don't seem so valid to me.

The observations about the linguistic limitations, the crudeness of some of the measures and cautions about the associations which the authors make, are examples of  some of the valid critiques, although they do not necessarily lead to the conclusion that this work does not help to some degree in trying to tease out the difference between lying and lying to ourselves - or, in the context of discrimination, between what we consciously tell ourselves and what what we may actually do, or what may be unconciously informing some of the decisions we make.  I think some of the critics may be asking that this research conclusively prove discrimination rather than that it show evidence which is suggestive of it or supports an inference in that direction.  Of course conclusive proof of most connections in the social sciences is unlikely to be forthcoming, moreover, that is not the test for admissibility for expert evidence in court. Observing that a particular approach does not answer every question you might have with respect to the problem it attempts to study does not mean the test is invalid any more than the fact that an MRI and a CAT scan offer different information means that either is invalid.  Finally,  terms like "disrimination" "bias," etc. are all themselves extremely loaded in that they might suggest more to some readers than the authors intend.

On the other hand, Daubert, the case that provides the framework for the admissibility of expert testimony,  does provide that expert testimony needs to meet some basic standards of reliability, etc.  that if all these criticisms are well-founded, would probably lead to exclusion. So, facts like the authors' own meta-analysis failing to account for the vast majority of the data is, if true, a significant omission from the book and one that it would be interesting to follow up on. Indeed, if you were planning to use such evidence in court it would be remiss of you not to follow up on it. 

I think other observations in the review are less valid. Take this one:  "But if a test gives results that are so far-fetched, it's time to start questioning the validity of the test."  First, "far-fetched" is an assessment that assumes the very thing that the authors are trying to discover, whether it is indeed "far-fetched" that someone might harbor unconsious racial stereotypes, no matter what their ethnic background. The comment assumes that racial heritage or background is some sort of invariable predictor of attitudes.  A casual glance around at the diversity of political opinions within various minority groups should dispell that idea. So the ethnicity of Malcolm Gladwell's parents, while surely not irrelevant to the question of his attitudes towards race, is probably not dispositive of them.

Second, that some data diverges from what you expect to find is not necessarily a reason to question the test. Sometimes it is. That is always one possibility. But it also is possible that it is a reason to question your assumptions about what is or is not "far-fetched."  There are a great many things which are true even though they are counter-intuitive, for instance that certainty is strongly correlated with accuracy.

Moreover, in this, as in so many areas of research that are highly politically charged, there are some hints that there may be ideological divsions presented as metholodgical ones driving some of the critiques.  There does seem to be a way in which normative disputes about the legitimacy of underlying assumptions sometimes morph into arguments that purport to be about rigor or metholodgy, with one side claiming that the other is not really rigorous, when they may be working from radically different foundational assumptions.  Whether this is going on here is a question that can only be resolved by further reading of the works on which the book is based and those critiques which the WSJ article references.  I do not know the answer to that question. Interested parties should definitely read it all and decide for themselves.

At the end of the day, this book is, as advertised, a presentation of the authors' research for a general audience, with all the limitations that go along with such attempts. That said, you need not endorse immediate adoption, by the courts or legislatures, of any of the implications of the authors' research here to conclude that it is interesting and worth further exploration. And, for better or worse, as the National Research Council's report from 2009 on forensic science illustrates, the validity of a practice has rarely been an insurmountable barrier to its acceptance in the courts.  So this research is likely to migrate there. (I think it may have done so already and look to more knowledgable readers to alert me to those cases.) Whether you want to use it or defend against it, may pay to give it a look. This book looks like one entryway to that exploration.

Comments

[Michelle Meyer]

Hi Tamara. Readers might also be interested in this WSJ review of Blind Spot by Daniel Levitin, which raises some important cautions and challenges for those who would rush to implement IAT research into law or policy just yet: http://online.wsj.com/article/SB10001424127887323829504578272241035441214.html. (Also, your link is bad; it should go here: http://thesituationist.wordpress.com/2013/03/03/blind-spot/)

[Tamara Piety]

Thanks Michelle for catching that! Will fix. And I agree caution is warranted. I think the full post at the Situationist notes that. But it is even better to supplement with the Levitin piece. Thanks!

[Brando Simeo Starkey]

You're far too kind. That WSJ "book review" was a hit piece.

For instance, the author writes, "When author Malcolm Gladwell took the IAT, it showed that he, the son of a black woman, is racist against blacks."

The IAT does not determine whether someone is racist. The author seemingly does not even understand the research he's knocking.

He also writes, "It turns out that the authors themselves published one in 2009, reviewing 184 independent samples and nearly 15,000 experimental subjects. The result: The IAT was very weakly correlated with other measures, failing to account for more than 93% of the data."

What does this even mean? What "other measures"? What "data"? Here is a summary of a work that Banaji and Greenwald published in 2009 which may be the article to which he is referencing: http://www.ncbi.nlm.nih.gov/pubmed/19586237 The summary does not back up any of his assertions.

[Tamara Piety]

@Brando well if I was "far too kind" it was because I haven't read all of this research, certainly not the link you offer, and so I guess "cautious" was the word I would use. It did strike me that the Gladwell example was particularly problematic for the reasons I discuss above and you point out another aspect of that. In the meantime, thanks for the cite to the meta-study. I hope that more people who know more about this topic than I will continue to weigh in.