We all want simplicity. The world is complex and we cannot deal with complexity all the time. We instinctively use heuristics to get through day-to-day life. Simplifying our thinking enables us to tell stories and frame theories. And as Daniel Kahneman noted (Thinking Fast and Slow, p. 201), “it is easier to construct a coherent story when you know very little.”
But not all simplifications are innate and not all are benign. We have a sorry history of replacing the complexity of an individual person by a single characteristic (Black, Gay, Woman, and so forth) and acting as if that is all we need to know. One of our great advances in recent times is in getting past this harmful form of oversimplification—not because such characteristics are irrelevant to anything at all, but because they can be used to hide rich and complex stories of persons or groups under a distorting cloak.
Simplification abounds in the world of legal education. We are living through a radical transformation brought about by a network of severe stresses and challenges. These include realignments in the legal services market, an increasingly anti-regulatory culture, a dramatic shift in the nature of preparation of individuals for law school, and much more. It is all very complicated and reduction to a simpler narrative is tempting. In the early days of this transformation, the main oversimplification was the search for perpetrators whom one could blame: Selfish faculty! Corporate mindset deans! A nitpicking and craven accreditor! And so forth.
By now, we are largely beyond this reductive obsession with blame. Yet, there are two broad oversimplifications that, as concerns law schools and legal education, get in the way of understanding, problem solving, and sometimes reasoned discourse.
One is the black-box view of law schools. By this I mean the tendency to see a law school as a machine for converting certain kinds of inputs (mainly LSAT scores) into certain kinds of output (mainly career and bar outcomes). On this view, a few inputs and outputs are all one needs to know about a school; specifically, to judge whether it is a “good” school or a “bad” school, and just how good or bad it is. What goes on inside the box is not a factor and is not of interest.
The poster child for black-box thinking is U.S. News. It notoriously judges law schools mainly on a limited number of inputs and outputs, and ignores what goes on within the school, except— bizarrely—how many dollars it spends. But U.S. News is not alone. A goodly number of bloggers and writers look only at inputs and outputs to draw sweeping conclusions about value and motive, usually negative (Bad school! Shut it down!). On this way of thinking, the character and quality of the education in the box is immaterial.
One problem with black box thinking is that it assumes the input and output data are good measures of something important. But it does not take much to see that the few measures commonly used are slippery and do not necessarily have an unambiguous meaning. Take LSAT profile. In an earlier post, I noted two factors (transfers in and out, and conditional admit programs) that suggest incoming LSAT profile does not have as clear a meaning as we like to think, and that we must take care in making inter-school comparisons. Here are three additional factors: (a) LSAT distribution is affected by incoming student diversity. Different groups have different LSAT distributions and so schools similarly situated can have different LSAT distributions because of different levels of student diversity. (b) Although LSAT is correlated with first-year academic performance, the strength of the correlation varies from school to school. Thus, two students, each with LSAT score of X, may have different likelihood of certain academic or performance outcomes, depending on the school they attend. (c) There are firms in the business of delivering increases in the LSAT scores of applicants. My understanding is that these increases can be substantial. But, to my knowledge, there is no data indicating whether these substantial increases also increase academic outcomes for the students—that is to say, whether the increase really increases the chance of success or just makes the student a better LSAT taker.
Similar points can be made about outcomes such as placement data and bar passage. Yet, even if we could have meaningful and unequivocal measures of inputs and outputs, a deeper problem would remain: that black-box thinking ignores most of what law school is about. Law school is all about educating students and transforming them into professionals. A law school is a complex educational enterprise delivering a wide range of educational services. Different schools have different strengths and weaknesses. Some are very strong in clinical education, others very strong in public policy. Some position themselves to provide opportunity; others position themselves to develop large firm lawyers. Some schools put much effort into developing professionalism and other non-academic competencies; others emphasize building scholarly competencies and future law professors. Understanding what law schools do inside the box is critical to understanding and evaluating them. That is why ABA site inspections involve not only mounds of paper, but two to three days of comprehensive on-site examination, to enable a close and careful look at what law schools do. We can never fully understand and assess a school if we restrict our attention just to a few imperfect measures of what goes in and what comes out.
Black-box thinking is given aid and comfort by another type of oversimplification, a cognitive bias, known as the halo effect. The halo effect is a tendency to attribute an evaluative characteristic (such as goodness or badness) to all aspects of a person or organization, based on a judgment or impression about one salient characteristic. This effect underlies first impressions about people (I like X, so he must be intelligent, so I will hire him) and the tendency to think businesses that are currently successful must be doing everything right (the premise of so many business books sold in airport bookstores).
The halo effect easily links with black-box thinking about law schools. For example, incoming LSAT profile strongly affects judgments about the school: high profile means good school, and thus means good program, good faculty, good teaching, good other things inside the box. But the halo effect in legal education goes beyond LSAT profile and other elements of the black-box model. For example, schools actively seek to build halos by hiring high-visibility faculty, on the quite logical theory that visibly good faculty translate into an overall good impression of the school, and through the halo effect support the conclusion that everything else about the school is good as well. Conversely, certain characteristics create a negative halo. A century ago, such a characteristic was being a part-time evening school, particularly one that served immigrant populations. Today, a negative halo is created by a law school’s taxable status: a school that is taxed is not liked within the law school world. Just as with other dislikes, this one triggers an assumption that anything else about the school must be bad, and that that is all one needs to know.
It is a little surprising to see black boxes and halos so widely used in discussions of law schools and legal education. As lawyers and educators, we are trained to ask questions, to appreciate the complexity of situations, to be able to see the multiplicity of perspectives, and to appreciate that there are few simple answers to hard questions. Black boxes and haloes are attractive because they allow us to tell a simple story with good guys and villains. But they impede our understanding and prevent us from seriously addressing the hard questions about law schools and legal education.
Law schools today do a wealth of different things and will likely do even more and different things in the years to come. Each school, in its own way, is dealing with stresses and changes; is trying to adapt and innovate; is trying to serve students and society in ways consonant with mission, strengths, and vision. There is an enormous amount we can learn about law schools, and an enormous amount we learn from the their experiments and experiences. Black box thinking and halos are comforting, but they disserve us by wishing away so much of what we very much need to know.