Adding Clarity to Law School Transparency

Lower tiered law schools that admit students with low LSAT scores should be celebrated for providing an opportunity to enter the legal profession for those students at the margins of society.  Unfortunately, their work has lately been characterized as predatory in popular media like The Atlantic and the NYT.  With its 2015 State of Legal Education report, Law School Transparency (LST) has instigated a dangerous national discourse, arguing that some ABA-accredited law schools have been engaged in a process of admitting students with low LSAT scores (which the LST labels “high risk”) who do not have a reasonable chance of passing the bar.  LST’s framing of these law school’s actions is disturbing given that: (1) the LSAT is at best a weak predictor of first-time bar passage, and (2) there is no evidence that law school graduates are not eventually passing the bar.

 

As a sociologist and law professor at Whittier Law School—one of the law schools that LST has labelled as high-risk—I decided to take a closer look at the results that LST relies on in coming to its conclusions about bar passage and risk.  LST relies strongly on results published in the LSAC National Longitudinal Bar Passage Study (LSAC Study).  I agree with the recent press release by the LSAC that it is problematic to use a 25 year old study to assess current bar passage risk.  Over the past 25 years, the LSAT has not only changed with respect to its score scaling, but also its content and substance.  However, even if you assume you can draw current inferences about today’s LSAT from the decades old LSAC Study, it fails to provide evidence supporting LST’s contentions.

 

LST clings to the false premise that the LSAT has a strong correlation to bar passage. 

This presumption is necessary in order to maintain a system of classification that categorizes a student’s risk of failing the bar based on her LSAT score.  Citing the LSAC Study, the LST 2015 State of Legal Education report overview, states that the LSAT is the best predictor before law school as to whether a student will pass or fail the bar exam.  But this statement is a bit misleading for two reasons.  First, it does not acknowledge that the indicators that do a better job predicting bar passage are those that appear after law school begins—i.e., a student’s law school GPA.  The LSAC Study states clearly that—of all the variables used in the study—cumulative law school GPA is the best predictor of first time bar passage.  This should come as no surprise.  What a student actually learns in law school should have significant salience in her first-time bar passage.

However, LST focuses on admissions standards.  Which leads to my second point as to why the LST’s statement is misleading.  The LSAC Study never stated that the LSAT is the best predictor of bar passage.  It states that of the factors that it has pre-selected, the LSAT is the best predictor of first-time bar passage.  It is important to state that the LSAC Study considered a few “before law school” variables in addition to the LSAT (for example, undergraduate GPA, college selectivity, law school status, socio-economic status (as measured by family income in high school and parents’ occupation), and race & ethnicity).  One could easily imagine that a factor like a student’s family income or financial resources available (a) when entering law school, (b) during law school, or (3) in the three months prior to sitting for the bar could be extremely probable and plausible factors that might not only correlate, but might actually cause some change in student performance with respect to first-time bar passage. It is vital to point out that the idea that the LSAT is a best indicator is relative, and in fact the LSAT might only be the best among a limited number of selected variables.

What LST fails to report is that while it is true that the LSAC Study reports that the LSAT is has a statistically significant correlation to bar passage, this correlation is weak at .30.  The LSAT as a single variable explains less than 10 percent of the variation in bar passage.  Only when the LSAT is used in a model with law school GPA does the correlation get stronger—and even then it is somewhat weak.  A majority of the variance (about 68%) in bar passage outcomes cannot be explained!  This means that how people actually perform on the bar varies a lot from how one might expect them to perform on the bar given their law school GPA and LSAT score.  We clearly need other factors to explain this variance.  In order to do this, it might make more sense to examine what happens to the bar taker after law school begins but before sitting for the bar—particularly exploring how students are preparing for the bar,  and analyzing their material and economic resources during prep period.

Instead LST’s overemphasis on the LSAT, treats the weak correlation between the LSAT and bar passage as somehow causally connected.  Granted, LST has never explicitly stated that there is a causal connection.  To their credit, they have stated that there are things, like increases in undergraduate GPA, that could offset a low LSAT score.  However, in the absence of this offset, LST treats the LSAT as having causal power—the LSAT either needs to be offset with higher undergraduate GPA or individuals with low LSAT scores and average uGPAs should be denied admission.  Even the authors of the LSAC Study warned against this crude thinking, and stated explicitly that the presence of a correlation does not mean that raising law school GPA or LSAT scores will lead to an increase in bar passage.

 

LST Overemphasizes First Time Bar Passage

This is less an issue of empirical debate and more of an interpretative discussion.  LST states that law schools are accepting unqualified students who can’t pass the bar, yet they offer no evidence that these students are not passing the bar on subsequent attempts.  You cannot solely measure passing the bar on whether students pass the bar on the first attempt.  That is unacceptable.

Of course every school should strive to increase their first time bar passage rate, for the sheer reason that multiple bar attempts can create economic and psychological hardships on graduates.  However, we should not overemphasize first-time bar passage so much that we ignore the fact that there are many graduates who pass the bar on subsequent attempts and enter a legal profession that either requires their bar standing and/or welcomes the advantage that their JD training provides.

According to the LSAC Study, 98% of bar takers with an LSAT score at or above the mean eventually passed the bar, compared to 90% of bar takers with an LSAT score below the LSAT mean.  There is likely evidence that eventual pass rates are well above 75% for low tiered law schools.  We know that one of the ways in which law schools with low first-time bar passage maintain their accreditation under the ABA is because they maintain a bar passage rate above 75% for those students who have graduated in the most recent five year period.  Eventually passing, whether on a second or third try—or within 5 years of graduation—seems like a far more important outcome than first-time bar passage, if we are seriously concerned with whether graduates are entering the legal profession.

 

Law school administrators and faculty must resist perpetuating and participating in the dangerous narrative advanced by LST.  Others have highlighted the paternalistic and problematic nature of this narrative.  In recent years, law school applications have been in decline.  This has made it somewhat easier for an individual to get accepted into a law school.  When admissions standards become easier, those at the margins of society—specifically the poor and black & brown racial and ethnic minorities—have the potential to benefit.  But just at the moment when it becomes easier for those at the margins to gain admission into the legal profession, the media and organizations like LST engage in a crusade to vilify law schools and pressure them into accepting fewer students.  This is done in the name of “transparency” and a paternalist drive to help the “unqualified” applicant who is at high risk of failing the bar.  Instead, LST’s strategy is at high risk of keeping black and brown people, and the poor from having the opportunity to ever sit for a bar. 

Comments

[[M][@][c][K]]

I think that no law professor - in particular one at Whittier law school should be allowed to make this sort of post without addressing the hard issues about their law school.

So since links tend to get caught in the SPAM filter, let's look at what Sheldon Bernard Lyke did not say when he mentioned "Whittier Law School—one of the law schools that LST has labelled as high-risk and the results he chose not to discuss, which are really quite shameful. And by the way, these really quite appalling outcomes are highlighted by LST, which is probably why it is a deeply unpopular organisation at the Faculty Lounge of Whittier.

Employment Score 25.8% - that is to say only ¼ of Whittier grads are successfully starting a career as lawyers on graduation - ONE QUARTER!! Search Whittier's bar passage rate for an independent 3rd party and it comes in for July and first time bar passage for 2014 was 42.7% less than ½! California as a whole was 69.4% In February 2015 the pass rate was an astonishing 30% - yep less than ⅓

But here is the thing - google "Whittier Bar Passage rate" and here is what pops up from Whittier itself:

"Taking the February 2014 General Bar Exam for the first time, 76.47% of Whittier Law School graduates passed. The overall passage rate for first-time takers of the February 2014 exam is 69% for California ABA-accredited schools, which Whittier Law School bested by more than seven points."

I mean hey, wow, Whittier is doing really really well on the California Bar Exam, according to Whittier.

Seriously, a professor at Whittier took time two write a screed accusing LST of dishonesty - I'm not sure that the administration of Whittier should be considered of sufficient moral character to be admitted to the bar.

Sheldon, you deserve vilification.

[Jojo]

"Dangerous," Sheldon?

A couple of thoughts. Why use the LSAT at all in admissions? What's the point in using it if not as a benchmark of academic potential and selectivity?

If it's largely irrelevant, why were schools more LSAT-selective in the past than they are now? Was the LSAT recently revealed to be irrelevant, changing the approach for law admissions for ever and ever amen from this point forward? I'd be more receptive to the argument that the LSAT is just one measure, and a weak one if there weren't recent NY State bar studies refuting it and if law schools had ever, ever acted that way in the past -- which we all know they didn't. Only when the well ran dry did you begin to challenge the LSAT. That is known in the oil industry as "talking your book."

Another comment not specific to the LSAT -
Law school profs are fine critics, but poor governors. On this issue, law schools, you are in charge. You don't get to pass the buck or pretend that LST is some powerful organization that you get to criticize. You are bigger, more powerful, and in charge vis-a-vis LST, so they get to criticize you. That's how it works. You can criticize tax policy, or Washington, or originalists, or living constitutionalists, or Republicans, or Democrats, or the Courts, or agency interpretations, or . . . , and be an outsider. Frankly, that's what most legal scholarship is -- criticism. That's fine, and there's a place for it. But when you are in charge and your conduct is scrutinized, you do not get to do ju juitsu to pretend that a tiny organization with one marginally paid employee but a powerful message is some big "paternalistic" threat to the rule of law.

[anon]

I stopped reading after the first paragraph when I figured out that “Lower tiered law schools that admit students with low LSAT scores should be celebrated for providing an opportunity” was not tongue-in-cheek.

Look SBL, it’s one thing for schools to take a risk on these students. But that’s not happening. Florida Coastal charged $43K last year with 46% of students paying full price. But FCSL had 0% risk of not being paid for any of those 106+ students they enrolled who had a 140 or lower LSAT and have a liklihood (statistically speaking) of never using their degree.

That isn’t virtuous on the part of FCSL. That’s a scam. Impose a risk—a financial risk, of course, because they obviously don’t care about prestige—on FCSL and I’ll read the rest of the article.

[Boby]

Brave post. I guess too nuanced for law school crits...

[[M][@][c][K]]

Brave - I think oblivious is a better word. ¼ of Whittier graduates secure legal employment, less than ⅓ to ½ pass the bar exam at the first sitting and the opening statement is:

"Lower tiered law schools that admit students with low LSAT scores should be celebrated for providing an opportunity to enter the legal profession for those students at the margins of society."

Really. You think it's brave?

Brave would be addressing the ¾ who never get enter the legal profession having paid $42,400 in tuition per year, a cost of attendance of over $70,000 p.a., who are borrowing $200,000 to have a 75% chance of not having that legal career.

[Shawn]

Several years ago I had the misfortune of being a professor at a TTT/TTTT law school, and let me assure you that it was predatory.

To the point where I quit working there after only one year rather than continue to be associated with it and also reported its shenanigans to the ABA.

Which did nothing, of course. Which is a big part of the problem.

[JM]

Sheldon - your school (Whittier) had the lowest bar passage rate of any California ABA school last July, with only 42% of first time takers passing. Regardless of your feelings about the LSAT as a predictor of bar passage, Whittier does a terrible job of identifying students with a strong likelihood of passing the bar nevermind having success in the profession. Coincidently (or not), the LSAT profile for Whittier's students is terrible. Your argument would carry more force if you were defying the odds, not proving LST right.

[Boby]

[M][@][c][K]--

1) Many students don't pay such tuition because they receive scholarships (or so I assume).

2) Most of the students pass the bar eventually--albeit the second time.

3) Many great students with low LSAT transfer after the first year.

4) Whittier, it seems, is one of the most diverse law schools in the nation. In California, where only 4.2 percent of Latinos are members of the bar--low tier law schools play an important role.

[twbb]

"Instead, LST’s strategy is at high risk of keeping black and brown people, and the poor from having the opportunity to ever sit for a bar."

And you accuse LST of paternalism?

If you want to provide the opportunity for members of disadvantaged communities to have the opportunity to sit for a bar, despite the high likelihood that simply by attending Whittier they will be unable to find a job as an attorney, then do not charge $42,400 a year in tuition. That is outrageous. And the tuition -- and your generous salary which it pays -- is the main issue here. It also makes your argument here very self-serving, which neither you nor any of the other law professors attacking LST have ever actually addressed. Because you personally benefit from the status quo, it is incumbent on you to better argue why we should agree with your position that the "opportunity to sit for the bar" is worth $42,400 of taxpayer funds a year.

[Reading]

Whittier (now) has a much better employment rate than it had in the past. According to the NALP report (available here: https://www.law.whittier.edu/resources/careerservices/NALP-2014.pdf) 65% of the 2014 class had JD required or JD advantage job.

[JM]

@ Reading - those figures are astonishing in light of a 42% July California bar pass rate for that same class.

[Reading]

@ JM--I see why. I guess because JD advantage does not require bar passage.

[JM]

@ Reading - I understood that. It is still a ridiculous claim. It must required near 100% of bar passers in JD required jobs, which is just ridiculous given the stats from much higher ranked schools like Pepperdine, Davis, Santa Clara etc.

[twbb]

I am with JM in thinking that there is something fishy about those numbers, but even accepting them at face value they do not support the decision to go to Whittier, particularly if you are someone at risk of not passing the bar.

The breakdown lists that out of the 194 students reporting for the survey, only 64 (or 32%) had bar passage required jobs. Assuming their sampling methodology is legitimate, that would mean that 75% of the students who passed the bar had JD required jobs. Not 100%, but definitely still suspiciously high for a school with Whittier's reputation and the glut of lawyers.

What's interesting is the median salary for JD-advantaged jobs is listed as $50,000. Sounds impressive, until you realize that the median salary for someone with less than 1 year of experience in the Los Angeles area is $50,759 (with many of those people likely having only bachelor's degrees). It also suggests that the school may be taking a very liberal reading of "JD advantage," and in any case does not seem to support the idea that $120,000+ of nondischargeable debt for a 65% chance at a $50-60k job is really worth it.

[confused by your post]

Professor Lykes,

Your post attacks the use of the LSAT as a means of evaluating law school applicants for admission. What methods of applicant evaluation do you propose as an alternative (other than an open admissions policy)? You are clearly aware of certain obvious items that are likely better predictors of bar passage than the LSAT. Unfortunately, useful predictors that manifest AFTER law school admission (i.e. applicant GPA in law school) aren't helpful to a law school admissions office evaluating applicants. Other useful predictors (race and parents' SES) are problematic.

It is easy to criticize something (especially if you have massive self-interest in doing so). It is harder to come up with solutions. We've heard your criticism regarding the LSAT. What predictors would you ask law schools to look at when evaluating law school applicants and how accurate do you feel such predictors are?

[Reading]

@ JM -- the report indicates 34% JD required job, while you mentioned 42% percent CA bar pass rate. So it means 80% of those who passed have secured a JD required job. Also, they must have graduates who passed the bar in other states. (Perhaps the employment figures include also those who took the Feb. bar?)

Additionally, it makes sense that there is a correlation between the numbers: likely the top students of the class are those who passed and also those who found job more easily.

[KayLa]

Sheldon,

You are obviously bright and accomplished. Like you, I taught at a low-ranked law school. I made all the arguments you make. You have to make those arguments to convince yourself that you are not part of the problem. I understand. It was a wonderful, rewarding, flexible job, and I did not want to leave.

At the end of the day, however, most of your students, like most of mine, are worse off after graduating from your law school. You can deceive yourself by keeping in tough with the handful of graduates that, by luck, connections, or brilliance, land lucrative or fulfilling legal jobs, but if you are honest, most of your students will never get a reasonable return on their investment. The vast majority of your students at Whittier are not given opportunity by their JD, but rather are given a financial burden that will crush them.

Again, you are smart. You can poke holes in LST's work. Congratulations. You have a PHD and a JD from elite schools; you should be able to do that, even with work from much more sophisticated people. But you should be seeking the truth. And do you actually think that the vast majority of your students are better off after obtaining their law degree from your school? Have you surveyed your graduates? Have you kept in touch with the ones who failed the bar multiple times? Again, you have a wonderful job, but at some point you may realize that most low ranked law schools create more burdens than opportunities.

[JM]

KayLa, that was a refreshingly honest perspective. I hope Sheldon responds.

[dupednontraditional]

@Shawn and @KayLa: as JM stated, thank you for your refreshingly honest perspectives. We would like to hear more of your (redacted) stories, should you be willing to share them. I'm sure that OTLSS and others would be happy to host.

[Anon]

I wonder if Professor Lyke can find anyone who agrees with his argument without having a financial incentive to do so.