We all want simplicity. The world is complex and we cannot deal with complexity all the time. We instinctively use heuristics to get through day-to-day life. Simplifying our thinking enables us to tell stories and frame theories. And as Daniel Kahneman noted (Thinking Fast and Slow, p. 201), “it is easier to construct a coherent story when you know very little.”
But not all simplifications are innate and not all are benign. We have a sorry history of replacing the complexity of an individual person by a single characteristic (Black, Gay, Woman, and so forth) and acting as if that is all we need to know. One of our great advances in recent times is in getting past this harmful form of oversimplification—not because such characteristics are irrelevant to anything at all, but because they can be used to hide rich and complex stories of persons or groups under a distorting cloak.
Simplification abounds in the world of legal education. We are living through a radical transformation brought about by a network of severe stresses and challenges. These include realignments in the legal services market, an increasingly anti-regulatory culture, a dramatic shift in the nature of preparation of individuals for law school, and much more. It is all very complicated and reduction to a simpler narrative is tempting. In the early days of this transformation, the main oversimplification was the search for perpetrators whom one could blame: Selfish faculty! Corporate mindset deans! A nitpicking and craven accreditor! And so forth.
By now, we are largely beyond this reductive obsession with blame. Yet, there are two broad oversimplifications that, as concerns law schools and legal education, get in the way of understanding, problem solving, and sometimes reasoned discourse.
One is the black-box view of law schools. By this I mean the tendency to see a law school as a machine for converting certain kinds of inputs (mainly LSAT scores) into certain kinds of output (mainly career and bar outcomes). On this view, a few inputs and outputs are all one needs to know about a school; specifically, to judge whether it is a “good” school or a “bad” school, and just how good or bad it is. What goes on inside the box is not a factor and is not of interest.
The poster child for black-box thinking is U.S. News. It notoriously judges law schools mainly on a limited number of inputs and outputs, and ignores what goes on within the school, except— bizarrely—how many dollars it spends. But U.S. News is not alone. A goodly number of bloggers and writers look only at inputs and outputs to draw sweeping conclusions about value and motive, usually negative (Bad school! Shut it down!). On this way of thinking, the character and quality of the education in the box is immaterial.
One problem with black box thinking is that it assumes the input and output data are good measures of something important. But it does not take much to see that the few measures commonly used are slippery and do not necessarily have an unambiguous meaning. Take LSAT profile. In an earlier post, I noted two factors (transfers in and out, and conditional admit programs) that suggest incoming LSAT profile does not have as clear a meaning as we like to think, and that we must take care in making inter-school comparisons. Here are three additional factors: (a) LSAT distribution is affected by incoming student diversity. Different groups have different LSAT distributions and so schools similarly situated can have different LSAT distributions because of different levels of student diversity. (b) Although LSAT is correlated with first-year academic performance, the strength of the correlation varies from school to school. Thus, two students, each with LSAT score of X, may have different likelihood of certain academic or performance outcomes, depending on the school they attend. (c) There are firms in the business of delivering increases in the LSAT scores of applicants. My understanding is that these increases can be substantial. But, to my knowledge, there is no data indicating whether these substantial increases also increase academic outcomes for the students—that is to say, whether the increase really increases the chance of success or just makes the student a better LSAT taker.
Similar points can be made about outcomes such as placement data and bar passage. Yet, even if we could have meaningful and unequivocal measures of inputs and outputs, a deeper problem would remain: that black-box thinking ignores most of what law school is about. Law school is all about educating students and transforming them into professionals. A law school is a complex educational enterprise delivering a wide range of educational services. Different schools have different strengths and weaknesses. Some are very strong in clinical education, others very strong in public policy. Some position themselves to provide opportunity; others position themselves to develop large firm lawyers. Some schools put much effort into developing professionalism and other non-academic competencies; others emphasize building scholarly competencies and future law professors. Understanding what law schools do inside the box is critical to understanding and evaluating them. That is why ABA site inspections involve not only mounds of paper, but two to three days of comprehensive on-site examination, to enable a close and careful look at what law schools do. We can never fully understand and assess a school if we restrict our attention just to a few imperfect measures of what goes in and what comes out.
Black-box thinking is given aid and comfort by another type of oversimplification, a cognitive bias, known as the halo effect. The halo effect is a tendency to attribute an evaluative characteristic (such as goodness or badness) to all aspects of a person or organization, based on a judgment or impression about one salient characteristic. This effect underlies first impressions about people (I like X, so he must be intelligent, so I will hire him) and the tendency to think businesses that are currently successful must be doing everything right (the premise of so many business books sold in airport bookstores).
The halo effect easily links with black-box thinking about law schools. For example, incoming LSAT profile strongly affects judgments about the school: high profile means good school, and thus means good program, good faculty, good teaching, good other things inside the box. But the halo effect in legal education goes beyond LSAT profile and other elements of the black-box model. For example, schools actively seek to build halos by hiring high-visibility faculty, on the quite logical theory that visibly good faculty translate into an overall good impression of the school, and through the halo effect support the conclusion that everything else about the school is good as well. Conversely, certain characteristics create a negative halo. A century ago, such a characteristic was being a part-time evening school, particularly one that served immigrant populations. Today, a negative halo is created by a law school’s taxable status: a school that is taxed is not liked within the law school world. Just as with other dislikes, this one triggers an assumption that anything else about the school must be bad, and that that is all one needs to know.
It is a little surprising to see black boxes and halos so widely used in discussions of law schools and legal education. As lawyers and educators, we are trained to ask questions, to appreciate the complexity of situations, to be able to see the multiplicity of perspectives, and to appreciate that there are few simple answers to hard questions. Black boxes and haloes are attractive because they allow us to tell a simple story with good guys and villains. But they impede our understanding and prevent us from seriously addressing the hard questions about law schools and legal education.
Law schools today do a wealth of different things and will likely do even more and different things in the years to come. Each school, in its own way, is dealing with stresses and changes; is trying to adapt and innovate; is trying to serve students and society in ways consonant with mission, strengths, and vision. There is an enormous amount we can learn about law schools, and an enormous amount we learn from the their experiments and experiences. Black box thinking and halos are comforting, but they disserve us by wishing away so much of what we very much need to know.
The meme here seems to be the idea that for profit law schools contribute to a glut of lawyers and that they lure people into law school with the promise they will practice law when they know that is not trued
But national data suggest the growth rate over the last two decades in licensed lawyers is pretty much in line with the growth rate in employed lawyers (actually I think the former has grown slower than the latter).
And getting a license has never meant that one will or wants to practice law. Again looking back over the last couple decades, less than half of all licensed lawyers are employed as lawyers. Hard to imagine that Charlotte is responsible for that data and hard to see how law school applicants can blame law schools for pulling the wool over their eyes for decades.
Of course in any particular region actual mileage may vary so national data may not rescue Charlotte, nonetheless, are we trying to knock down a straw man?
Posted by: Anon | January 03, 2015 at 08:48 PM
"But national data suggest the growth rate over the last two decades in licensed lawyers is pretty much in line with the growth rate in employed lawyers (actually I think the former has grown slower than the latter)."
Isn't that a "duh" sort of observation, Anon? After all, employed lawyers need a license, no? What's the point of this observation?
"looking back over the last couple decades, less than half of all licensed lawyers are employed as lawyers"
Here is another sort of irrelevant observation, no? What is your point? That one half of newly minted graduates of law school intend to pass the bar and obtain a license, but don't intend to practice law? What's your support for that sort of obviously wrong conclusion?
If your point depends at all upon the notion that graduates of law school expect, generally, to pass the bar and get a license - and, although hard to discern it seems that getting a license is at least one of the suppositions supporting whatever point you are attempting to make - then your comment, whatever your point, misses the point of this thread entirely.
Posted by: anon | January 03, 2015 at 09:24 PM
Law schools are in the business of producing licensed lawyers. If law schools aren't producing licensed lawyers at a rate higher than lawyers finding work as lawyers then law schools aren't creating an oversupply problem.
(And from what I can tell from a quick look at the data, over the last two decades the number of licensed lawyers increased about 33% while the number of employed lawyers increased 39%. In other words it has become easier to get a job as a lawyer over that time period.)
As to the point that only half of licensed lawyers work as lawyers I can't obviously conclude that law school applicants don't intend to practice law but it seems hard to sustain a meme that law schools are fooling them about the odds of practicing law in the face of decades of data indicating that the JD leads to all sorts of non-practice careers. (And as the data on JD incomes shows these are quite lucrative relative to going through life with just a BA.)
Posted by: Anon | January 03, 2015 at 09:37 PM
"as the data on JD incomes shows these are quite lucrative relative to going through life with just a BA"
o rly?
care to share this data?
Posted by: No, breh | January 03, 2015 at 10:09 PM
no, breh: I guess you be a newbie here, eh? well here you go anyway:
http://www.jstor.org/discover/10.1086/677921?uid=3739560&uid=2&uid=4&uid=3739256&sid=21105556479493
Posted by: Anon | January 03, 2015 at 10:18 PM
No, breh
Anon is likely relying on the infamous S&M paper. 'Nuff said about that. Reading that paper with all the caveats, assumptions and nuances yields the inescapable conclusion that it did not even approach a meritorious and rigorous examination worthy of the hype some afforded it.
As for the blurry nature of Anon's "points," Anon seems to be saying:
1. There are more practicing attorneys than licensed attorneys (over the last two decades the number of licensed lawyers increased about 33% while the number of employed lawyers increased 39%),
2. Law schools are therefore "not creating an oversupply problem" and
3. Only half of licensed attorneys are lawyers.
Points one and three are contradictory (if "employed lawyer" = employed as a lawyer); in any event, these "points" are approaching the appearance of intentional misstatements intended to derail the subject of this thread. Accordingly, I'll leave it to others to decipher Anon's good faith. The "points" Anon is attempting to make are at best irrelevant, and at worst, deliberately wrong.
Posted by: anon | January 03, 2015 at 10:36 PM
No I did not say there were more practicing attorneys than licensed attorneys - the opposite is true, in fact. And it has been true for at least the last two decades.
And that is the point.
Posted by: Anon | January 03, 2015 at 10:51 PM
Anon, you are missing the fact that many lawyers give up their license when they are unable to find a job that requires it. But if the quantitative data doesn't convince you of the severe JD oversupply issue, and the hordes of JDs publicly speaking about how law school was the worst choice of their life, and the exploding debt levels, and the continuing drop in reported lawyer satisfaction don't convince you that something is wrong, then nothing will.
Posted by: twbb | January 03, 2015 at 11:36 PM
Twbb - data or speculation? Hard to believe that number would outweigh the correlation between licensed lawyers and working lawyers over many years.
Posted by: Anon | January 03, 2015 at 11:59 PM
The numbers, Anon, are really, really bad, especially at schools like Charlotte. The only way you can spin it as a positive is if you make unwarranted assumptions about what you think graduates are doing. The actual recorded employment rates tell a very dismal story.
Posted by: twbb | January 04, 2015 at 12:55 AM
@Anon/9:37 p.m.:
If it has actually gotten easier in recent years to find work as a practicing attorney, why have so few law schools over the last six years managed to place than 60% of their graduates in full-time/bar-required jobs within 9 months of graduation and 4 months of bar results being published in most states?
Posted by: John Thompson | January 04, 2015 at 09:15 AM
Interesting that the Dean discounts quantitative measures. I know that Infilaw is big into Six Sigma, or at least was. Some of the deans at other Infilaw schools were applying it to their operations. I would assume that was a corporate mandate.
Posted by: Just saying... | January 04, 2015 at 12:28 PM
Jay, Are you not going to respond to the challenge to disclose Charlotte's data for low-LSAT students? Nice tap-dancing, but we need data.
Posted by: AProf | January 04, 2015 at 01:37 PM
twbb - when you say the "actual recorded employment data tell a dismal story" what data are you referring to? my understanding is that except for one down year in 2008, there have been more people working as lawyers every year for more than the last decade at least and earning more income every year even in 2008.
Posted by: Anon | January 04, 2015 at 02:55 PM
Anon, the absolute numbers is not very relevant and I am puzzled as to why you would even appeal to that. The RATE is what is important, and 36% of your law graduates not finding work for which bar membership is required is a terrible number. Considering this does not include low-paying, no-advancement jobs like document review, or people who start solo practices out of desperation, that number is horrifying. Have you asked yourself why you are trying to measure law school employment outcomes by metrics that nobody else uses? Do you think maybe that this is a sign that you are doing something wrong?
Posted by: twbb | January 04, 2015 at 05:22 PM
2:55pm - it may be true (may!) that more people are working as lawyers each year. But the number that matters is the number of graduates vs the number of jobs filled each year. So sure, great if 10k more lawyers are out there now than 5 years ago, but if there have been 200k graduates there's a problem. (I know these are not exact numbers).
Earning more income? Well sure once you factor out all the grads who never became lawyers that statement may be true. But I think inflation alone would make that statement true in many professions.
Posted by: Anon | January 04, 2015 at 05:28 PM
re: the Anon comment w/r/t S&M paper after my comment above, I've actually posted on the problem with that paper before, either here or elsewhere, which I'll summarize as: bad theory, bad assumptions, insufficient and/or improper controls, correlation/causation fail, comparing apples & oranges, and past performance does not guarantee future results.
In short: A JD does not magically lead to improved non-legal career opportunities, and I've not once seen a good, logical explanation of why it should or would.
Posted by: No, breh | January 04, 2015 at 05:29 PM
re: the Charlotte jobs data, good lord that is painful:
http://www.lstscorereports.com/schools/charlotte/jobs/employers/2013/
Around 13% school-funded too....jeez....
Posted by: No, breh | January 04, 2015 at 05:32 PM
That's pretty depressing. And of course, "business" probably does not mean investment banker or consulting job.
Posted by: twbb | January 04, 2015 at 06:07 PM
twbb: not sure where you got the 36% figure but here is what my quick and dirty review of the data I could find show:
1) over the last 20 years or so (and probably going back for longer than that) the number of people working as lawyers is just under 50% of the number licensed as lawyers.
In other words it has been true for many years that only half of all JDs practice law.
Even if one does not accept the conclusions about the value of a JD demonstrated by Simkovic and McIntyre (whose research has been vetted by many scholars no doubt more qualified than "no,breh"), it seems a big stretch to argue that law schools have been fooling people for decades about where JDs end up working.
And certainly since this is a longstanding correlation it makes no sense to conclude that law schools started doing something differently that caused a temporary mismatch in supply and demand - it is my understanding that the critics base their "fraud" claims on law school statements about employment that reach back many years.
2) The data also show that in shorter time frames temporary mismatches in the production of licensed lawyers and people employed as lawyers show up. But they go in both directions. So, for example, from 1998-2002 the number of licensed lawyers increased by 6% but the number of people working as lawyers increased by 17.7%. From 2003-2007 the number of licensed lawyers increased by 8% and the number of people working as lawyers increased by 7.6%. Finally, from 2008-2012 the number of people working as lawyers increased by 5% while the number of licensed lawyers increased by 7%. In other words, when the mismatch was greatest it was because of an underproduction of licensed lawyers.
Of course, none of this makes any difference for a particular individual caught in the downdraft after 2008 but it does undermine the claim that law schools have been over-producing licensed lawyers.
Posted by: Anon | January 04, 2015 at 06:19 PM