2019 ABA Bar Pass Spreadsheet Analysis, Part 1 - 2018 Bar Pass Rate

There have been several articles published recently regarding the 2019 ABA Bar Data Consumer Info Spreadsheet (the “Spreadsheet”), including one here on The Faculty Lounge by the estimable Gary Rosin.  The Spreadsheet provides bar data through the July 2018 bar exam regarding all ABA accredited schools.  The data includes the 2018 first time pass rate, both overall and by jurisdiction, for each school.  The data also includes the “ultimate” bar pass rate (UBP) for calendar year 2016 graduates within two years of graduation. This is the second year the ABA has published this consumer data in this form.  Last year I published a four-part series of articles here on the Faculty Lounge about the inaugural spreadsheet.  (See here, here, here and here.) Most of the articles about the new spreadsheet, (other than Gary’s, of course) have simply recited some overall figures and identified a few of the best and worst performers without any real analysis of what the figures mean.   In this series of posts, I will try to extract some meaningful lessons from the raw data in the Spreadsheet.  I will start by analyzing the 2018 first time bar passage rates, then, in a subsequent post or two, delve into the UBP rates for 2016.

The overall first-time bar pass rate in 2018 was 74.82%, down from 77.34% in 2017.  This is not particularly surprising given the overall weakness of the entering class of 2015.  2015 was the low point for law school admissions with many law schools admitting high numbers of weak students to partially offset five straight years of declining applications.

As usual, by and large, bar outcomes tended to very closely track the entrance credentials of students three years earlier.  However, there were a few notable bright spots, schools which substantially outperformed the predictors.  Four schools in particular stood out for their first-time bar pass performance in 2018:   Liberty U. (44/46, 95.65%); Belmont (73/78, 93.59%); Cleveland State, (76/83, 91.57%) and Florida International (129/147, 87.75%).  Kudos to these schools!

Unfortunately, many other schools did not fare nearly as well.  (To find out who really stunk it up in 2018, continue reading below the fold.)

Which law schools had the worst bar performance in 2018?

The following chart depicts the bottom 25 performers among all U.S. law schools (excluding Puerto Rico’s schools) as measured by first-time bar passage rate.  All 25 schools were below 52%.  All other law schools had a first-time pass rate of at least 55% or higher.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Why did graduates of these schools do so poorly on the bar exam?

While there is a tendency to assume that law schools with bad bar pass rates are "bad law schools" in the sense of having bad teachers, or having an insufficiently rigorous curriculum, or having inadequate academic support and bar preparation programs, this is not necessarily the case.  Rather, the primary factor is the school's admissions policies, not the quality of legal education offered by the school.  As you can see on the table, with the exception of four California law schools, University of San Francisco, Southwestern, McGeorge and Cal Western, every law school on the bottom 25 list had very high risk admissions policies in place for their fall entering class of 2015 (when most of the 2018 first-time bar takers matriculated into law school).  By very high risk, I mean that at least 25% of the students that matriculated were at very high risk of failure, and at least 50% were at high risk of failure, based on the LSAT Risk Bands table I first published in 2014, and which has been repeatedly validates since.  I have argued that admitting any significant percentage of extremely high and very high risk students is unethical, irresponsible and violates ABA Standard 501.  At the very least, it is a recipe for disastrous bar passage rates.  Perhaps not surprisingly, as the chart also depicts, many of the schools on the list have either gone out of business, been placed on probation, or been sanctioned or had some form of remedial measures imposed upon them by the ABA at some point since 2015 (although some schools were later found back in compliance).  

There were 14 mainland US law schools that admitted classes with at least 50% very high risk students in 2015, that is, with a median LSAT of 146 or lower (not counting Charlotte, which is out of business and did not report results for 2018).  11 of the 14 had first time bar pass rates below 50%.  The three that didn't were Florida Coastal (56.76%),  Faulkner (55.27%), and North Carolina Central (75.59%).   But all three of these schools had extremely high attrition rates from their first year classes.   Florida Coastal's 1L attrition in 2016 (for the 2015 entering class) was 38.6%.  North Carolina Central's was 34.3% and Faulkner's was 21.8%.  The ABA presumes that first-year attrition rates above 20% indicate exploitative admissions practices.  Both North Carolina Central and Florida Coastal were subjected to remedial measures by the ABA in part because of high attrition rates.  Four law schools admitted classes where at least 50% of the class were at extremely high risk of failure (median at or below 144): Thomas Jefferson, Arizona Summit, Western Michigan (formerly Thomas Cooley) and Texas Southern.  All, predictably, had first time bar pass rates of 45% or lower, despite also having very high first-year attrition rates.  Thomas Jefferson's 1L attrition rate was 47%(!), Arizona Summit was at 33.5%,  Texas Southern was at 21.3%, and Western Michigan was at 13.9%.

When one considers the combined effect of academic attrition and bar pass failure, it becomes even clearer how catastrophically bad the performance of these very high risk classes really are.  Whittier's 2o15 entering class had 42.5% 1L attrition and still achieved less than a 22% first time bar pass rate.  Golden Gate had a 39.3% attrition rate and a 35.90% bar pass rate.   To put this in even starker perspective, consider the following chart, depicting the number of students that started law school in 2015 compared to the number of graduates who passed the bar on their first attempt 3 years later:

School	Matriculated 2015	First Time passers in 2018
Whittier	147	19
Golden Gate	117	28
Thomas Jefferson	247	37
Arizona Summit	267	44
Western Michigan	448	144
Appalachian	33	9
La Verne	95	17
U San Francisco	205	55
Valparaiso	130	34

While this chart doesn't account for the possible effects of transfer attrition from the top, it is still appalling that there are law schools accredited by the ABA where fewer than a third of students complete law school and pass the bar on their first attempt.  Of course, some students who failed the first time will eventually pass, but the sad truth is that well over half of the students that started law school at many of the bottom-feeder schools will either never earn a JD or, if they do, will never pass the bar.   That is shameful.  The ABA should establish a presumption that an LSAT median at or below 146 is not in compliance with Standard 501.

Another important point about this group is that 21 of the 25 schools had a first-time pass rate more than 15% below the weighted state average of the jurisdictions where their graduates took the bar, so they did not just do poorly relative to the rest of the country, but also did poorly relative to their in-state peers.  The four exceptions were all in California where the overall first-time pass rate for ABA graduates in 2018 was only 60.86%.  Schools within 15% of this rate were Western State, Southwestern, McGeorge, and Cal Western.  Three of these schools (Southwestern, McGeorge and Cal Western) did not have high risk admission practices. Of the California schools on the list of the bottom 25, one school, Western State, outperformed its predictors, while another school, University of San Francisco, woefully underperformed.  (I will discuss the implications for California in more detail in a subsequent post.) 

The key takeaway from the 2018 first-time bar pass rates is this:  VERY LOW LSATS = VERY LOW BAR PASS RATES.  No law school has cracked the code on how to turn very low LSAT scorers into first-time bar passers on a consistent basis. Responsible law schools should be extremely cautious in admitting any students with an LSAT at 146 or below, taking only those students with substantially better than average grades, and should follow North Carolina Central's decision to never admit anyone with an LSAT below 142.  Students with an LSAT score at 146 or below should strongly consider another career, even if you are admitted to a law school.  Although some students do succeed with LSATs at 146 or below, more than half will never realize their dream of becoming a practicing lawyer. 

Comments

[Derek Muller]

Thanks for your analysis as usual. I want to push back, however, on one point, and that's attrition. I think it's very important to remove transfer attrition from the overall attrition rate when considering the deficiencies of schools. I know your chart says it doesn't account for it, but I think it really needs to be emphasized. Transfers, I think, are the *most likely* cohort to pass the bar exam. That is, other schools look at these students, usually the very best of any given law school, and take them as their own. So Thomas Jefferson had 102 students (47%) attrition, but more than a quarter of those (28) were transfers. Whittier had 62 students (42.5%) attrition, but more than 40% of those (26) were transfers. Arizona Summit had 99 students (33.4%) attrition, but about a third of them (35) were transfers (and likely the vast majority of them to Arizona State). In short, higher-ranked schools actually help perpetuate the cycle of bar exam dysfunction at the most marginal law schools when they accept so many transfers from these schools. Now, that isn't to say the admissions practices are perfect (or even necessarily acceptable), or that the academic attrition isn't significant, as you note. But it's to say that, I think, transfer practices of higher-ranked law schools tend to distort (I admit, to a degree, and not in totality) the bar passage rates of some of the most marginal schools, particularly those with high *overall* attrition, in ways that really need to be materially evaluated.

[William W Patton]

Dear David,

You are incorrect when you state that "the primary factor [regarding bar exam passage rates] is the school's admissions policies...." Actually, the factor with the greatest impact is the state's bar exam cut score. Let me provide one comparison from the 2018 bar passage data to prove my point:

Mercer: LSAT (154/152/150) GPA (3.49/3.32/3.06) Passage Rate 73.7% Mean MBE 137.7
Southwestern (154/152/149) (3.39/3.17/2.91) 53.0% 140.9

The entering student statistics for these two schools are very similar, but there is a 20.7% difference in bar passage rates based on the difference in state cut scores (135 vs. 144 for California). But two other statistical comparisons are equally important. First, even though the student credentials for these two schools are similar, Southwestern students outperformed Mercer students on the MBE by 3.2 points. In addition, Southwestern scored higher on the mean MBE even though it had a much higher student diversity percentage (Mercer 18.4% versus Southwestern 30.3%).

This comparison demonstrates not only that Southwestern is providing its students with a "value added" education (mean MBE scores higher than their predictive index and higher than the national mean MBE score), but that the ABA Council's proposed 75% bar passage rate in 2-years of students' graduation is a terrible idea. That proposed standard will de-accredit schools like Southwestern that have both high diversity and above the national mean MBE scores just because of state politics (legal monopoly assured by unrealistically high cut score).

I look forward to your next installments and your discussion of the dilemma and possible solutions for the high cut score/accreditation problem.

[David Frakt]

Derek -

Good point. I usually try to use non-transfer attrition when analyzing specific schools. But while losing the best students to transfer attrition may depress a school's bar passage somewhat (assuming that those who transfer will pass at a very high rate), whatever group of students who remain at a law school should still be able to achieve a 60% or higher first-time pass rate. The problem is with law schools admitting large groups of students that they have every reason to believe are more likely to fail than pass. Perhaps one reason that good students at marginal schools leave is because they find that they are surrounded by so many weak students and they want to be at a school where they will not only have better job prospects, but will be surrounded by brighter, higher-achieving classmates.

[David Frakt]

William -

Thank you for your comments. You are right that the cut score has a large impact on passage rates, especially when comparing schools from jurisdictions with very different cut scores. But law schools know perfectly well what the cut score is in their jurisdiction and what kind of bar results they can expect based on the entrance credentials of whom they admit. (If they don't know, I would be happy to be hired as a consultant and I can explain it to them.) What I find troubling is that even though they know, many schools still admit droves of students who are destined to fail.

It seems to me that your real beef is with the California bar, which you and I both believe has set their cut score artificially high. But despite the unnecessarily high cut score, the latest numbers demonstrate that, even in California, law schools with reasonable admissions policies can still achieve a 75% or higher UBP within two years. You are incorrect to state that the proposed standard would "de-accredit schools like Southwestern." Southwestern had a 79.55% UPB for 2016, despite only having a 38.26% first-time pass rate in 2016. Similarly, UC Hastings, which had a disastrous 2016 first time pass rate for a UC school at 52.30%, had a UBP for 2016 of 82.3%. As I will make more clear in a forthcoming article, the only schools that have reason to fear the proposed rule are those who are admitting high numbers of unqualified students, not those which have high diversity. If the rule forces those schools to be more responsible in their admissions polices and stop taking very high and extremely high risk students, then I am all for it.

[William W Patton]

David, I think that your predictions for 2018 ultimate bar passage rates based on the 2016 ultimate passage rates overestimates the number of California ABA schools in 2018 that will meet the 75% in 2-year proposed standard. Let's use Law School Transparency's own predictive model that states that a school with a first-time passage rate below 60% and repeater passage rates of less than 30%/15%/7.5% for three successive repeater bar exams is a school at risk of failing the proposed standard. Consider three California schools under the LST model:

In 2018 USF had a first-time passage rate of 33% [well below the 60% predictor] and its Feb/July 2018 repeater rate was only 24% [below the 30% predictor]. Under the LST predictive model USF is an at-risk school. It is also a school with high diversity (the 2018 entering class was 30% Hispanic and Black).

In 2018 McGeorge had a first-time passage rate of 50% [below the 60% predictor] and a Feb/July 2018 repeater rate of 23% [below the 30% predictor] and is under the LST test an at-risk school. McGeorge had a 2018 entering class with a 23% Hispanic and Black students.

In 2018 Southwestern had a first-time passage rate of 53% [below the predictive 60%] and a Feb/July 2018 repeater rate of 28% [below the predictive 30%]. It had a 2018 entering class with 31% Hispanic and Black students.

The reason that California schools that met the 75% in 2-year standard in 2016 are now at risk for 2018 graduates is that the repeater passage rates in California have been rapidly falling. For example, from 2014 to 2018 Southwestern's repeater rate has fallen 13%; McGeorge's repeater rate has fallen 21%; and USF's repeater rate has fallen 16%.

Making matters worse, the NCBE has recently predicted that repeater bar passage rates will fall for the next several years. Therefore, according to LST's predictive model, many more California law schools may fail the 2018 ultimate bar passage 75% in 2-year standard than failed that standard in 2016.

On another point, it appears that you and I disagree on the definition of what constitutes "high diversity". I include within that term the many California law schools that have diversity rates above 20%. But the effect of the 75% in 2-year standard on diversity cannot be adequately viewed by simply looking at the effect on individual schools. In California, if we use the LST predictive model, there are 8 or 9 schools at risk of failing that model for 2018 ultimate passage rates. If we look at the number of 2018 Hispanic and Black first-year students in those California ABA schools that translates to the loss of 59% of Hispanic first-year students (497/842) and 51% of Black first-year students (105/205).

Again, I look forward to your next few installments.

[David Frakt]

William -

So it turns out that LST's predictive model is not very accurate for California, although it works pretty well around the rest of the country. As I will elaborate in a forthcoming post, even though California's first-time bar pass rate is nearly 15% below the national average, the UBP for California essentially matched the national average for the class of 2015 at 88%, and was just 3 percentage points below the national average for the class of 2016, at over 85%. And this was probably largely attributable to the decline in admission standards from the entering class of 2012 to 2013. So, even though California's high cut score results in fewer graduates passing the bar on the first go-around, it has very little impact on the UBP. At California schools with low first-time pass rates, a significant percentage end up passing on repeated attempts. At some schools, more students have passed on subsequent attempts that on the first attempt. It turns out that students whose predictors suggest they should be able to pass a bar, do end up passing the bar in California, it just takes them a little longer. As I hope to demonstrate with my upcoming post, what is true for the rest of the country is also true in California - only law schools with irresponsible admissions policies that admit large numbers of very high risk students have anything to fear from the proposed 75% UBP standard. Your dire predictions for diversity grossly exaggerate the risk.

There is a basic math problem with your calculations. The repeater rate that you cite is the repeater rate for all students from a particular school, it is not the rate for those retaking the test for the first time, or the second time, or the third time, but rather an average of all students retaking the test no matter what number attempt it is. So, a 28% overall repeater rate likely means a much higher rate of passage for first time repeaters. The reason that the numbers for these schools in on the decline is that many California law schools dramatically lowered their admissions standards gradually starting in 2011 and continuing through 2015. Things leveled off for the most part in 2016, although a few school continued to decline. So the weakest graduating classes were 2017, 2018 and the upcoming 2019 graduating class. These classes are likely to have low first time bar pass rates and low repeater rates compared to their predecessors. But the ABA should not set a standard based on the worst entering class in history. The whole point of having a meaningful standard is to push schools to have both reasonable admissions standards and to make sure that they are adequately preparing students for the bar exam. That means the standards should be based on what is reasonably achievable by schools with responsible admissions policies.

[William W Patton]

David,
You indicate that California’s 2016 UBP passage rate “was just 3 percentage points below the national average for the class of 2016, at over 85%. And this was probably largely attributable to the decline in admission standards from the entering class of 2012 to 2013.” What data are you relying on for this conclusion? The data regarding the lowest performing California law schools demonstrates a modest reduction in admission statistics from 2011 to 2013. For instance the median LSAT/GPA’s for those schools for 2011, 2012 and 2013 were: (1) Golden Gate (3.05/3.12/3.11); (152/151/150); (2) California Western (3.21/3.18/3.13) (153/151/151); (3) McGeorge (3.40/3.33/3.24) (158/156/154); (6) Southwestern (3.21/3.26/3.17) (153/153/152); (5) USF (3.40/3.26/3.28) (157/156/153); (6) Whittier (2.95/2.92/2.95) (152/151/149); (7) La Verne (3.05/2.78/2.83) (153/151/147); (8) Thomas Jefferson (3.01/2.93/2.86) (151/148/146); (9) Western State (2.95/3.12/3.09) (151/151/150).
As this data demonstrates, there was no dramatic overall decline in these CA schools’ LSAT/GPA’s during 2011 to 2013. Only one law school, TJ in 2013, had scores in your “very high risk 145-146” range, and only three law schools, Whittier, La Verne, and Thomas Jefferson, had scores in your “high risk 147-149” range in 2013.
The CA schools’ LSAT/GPA declines from 2011 to 2013 are consistent with the national declines among almost all ABA schools. In addition, these schools’ declines in their matriculants 25th percentile were much lower than the national reductions for those years. For instance, according to Professor Jerry Organ, from 2010-2013 the percentage of matriculants in the LSAT range of 145-149 increased by 8% and scores below 145 increased by 56.8%. Therefore, these CA law schools’ increases of matriculants in the lowest LSAT bands actually increased at a much lower rate than the national matriculation patterns.
Further, the school with the largest drop in mean matriculants’ LSAT scores was La Verne which fell from a mean of 153 in 2011 to a mean of 147 in 2013. However, that decline had almost no effect on the CA schools’ 2016 ultimate bar passage rate because in 2013 La Verne only admitted 49 students.
As far as I have been able to determine at this point, the slight decline in the quality of these CA ABA schools’ 2012 and 2013 matriculants’ LSAT/GPA’s does not explain why CA law schools’ 2016 ultimate bar percentage rate fell 3% below the national average.

[anon]

As usual, these folks are arguing past each other, and ignoring common sense.

Perhaps the overall decline in LSAT scores is attributable to better ranked students choosing other options.

And, the elephant in the room is that these folks praise law schools when they do better than the stats would suggest, but never, ever, not ever ever will blame faculties for underperformance.

In other words, the heroic faculties can do no wrong. If the students do better than expected, that result is attributable to faculty excellence. If the students do worse then expected, then other factors are to blame (preposterously, solely LSAT scores).

The truth is that some faculties do better than others. To deny this is just evidence of a mind-set that rejects common sense.

When a school fails to produce graduates that perform at the incredibly low level expected, and then temporarily corrects that failing, then some are willing to attribute that improvement to faculty effort. When that same school then slips back down into the sloth-ridden, lazy, arrogant ways that led to the first failure, and within a decade or so, those same folks refuse to use their common sense, and search in vain for an answer that does not implicate faculty incompetence.

David's work is ok, so far as it goes. But, he can't admit that faculties play any role in all this. Perhaps that is because he is so invested in blaming it all on LSAT scores. Or, perhaps his own interests are involved. One can't say.

What one can say is that faculty performance plays a role: if faculties take (or are given) credit when their students do better than expected then they should take responsibility when they do worse.

[anon]

to wit:

"However, there were a few notable bright spots, schools which substantially outperformed the predictors. ... Kudos to these schools."

Why "kudos" ("praise and honor received for an achievement")?

Are you saying faculty competence had something to do with "outperforming the predictors" at those schools?

[Scott Fruehwald]

To anon: FIU had the highest bar pass rate on the last few Florida bars. They did this through a program that concentrated on student learning.
See https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2960192

So, yes, a dedicated faculty can make a huge difference.

[anon]

And, so, can a slack faculty make a big difference?

[anon]

Silence.

Those who feed off law schools won't say that faculty incompetence is a factor.

Faculty KUDOS ... of course.

Again, common sense. If a law student did poorly, improved when disciple was applied (e.g., probation) and then slacked right off after discipline was removed (e.g., probation lifted), then law school administrators wouldn't hesitate even a moment to draw a conclusion.

Frakt posts and posts and posts about LSAT scores. But he won't consider the obvious fact that law faculties can and do make a big difference, and that big conclusions can and should be drawn about faculties that are repeat slackers.

If a law school was found by the ABA within the last ten years or so to merit probation, and effected a brief period of improvement, but now -- already! -- has fallen right back into a dismal and unacceptable level of preying on young lives, then that law school should be singled out.

Law school observers like Frakt should be advocating, in addition to admission standards based on LSAT scores, for an examination of the contribution of a faculty culture of stubborn arrogance and incompetence to blatant violations of ABA standards (which of course, mainly go unaddressed, other than by weak and ineffectual "admonitions" that don't even appear on the law school target's website).