Wait a Minute! Let's Think About the Mathematics of Bar Passage

For most states, you must graduate from an ABA accredited law school to be eligible to take the bar exam at all. As Florida has been the main topic of conversation in the current discussion, I will use it as an example. Unless an applicant has been in practice for ten years in another jurisdiction, all applicants for the Florida bar must attend an ABA school. See Florida Rules of the Supreme Court Relating to Admissions to the Bar 4-13.1(a) & 4-13.2. The out-of-state practitioner rule seems to be new as passage rates in this category only exist since 2015.

Florida had a 68.2% success rate on the bar last July. See Florida Results. (The success rate for practitioners is just slightly better at 68.9%). All of the non-practitioners—and probably most of the practitioners, too—are graduates from ABA schools.

It is not possible for all ABA schools to achieve an 85% (or even 75%) pass rate in Florida. Despite what is said on A Prairie Home Companion, all of our children cannot be above average. The reality is that if one ABA school gets a passage rate that is above the state average, another one will be below it.

Further, the passage rate trend on the Florida bar raises an important question. As the table and chart of July pass rates below indicates, the pass rate on the Florida bar has been declining fairly dramatically over the last eight years:

Year        Overall    Practitioners

2009        80.0        —
2010        79.2         —
2011        80.1          —
2012        80.2         —
2013        77.2          —
2014        71.8          —
2015        68.9        69.6
2016        68.2       68.9

This declining passage rate can have two sources. One contributing factor, as much of the discussion about bar passage rates have assumed to be the sole factor, is that the quality of the people taking the bar exam has declined. The other possible factor that the discussion has ignored is that the bar exam itself has become more difficult.

There is some support for this second factor being important in the percentage of practitioners who pass. There is not a significant difference between that rate and the overall rate. All of the practitioners presumably graduated from law school at least a decade ago and before the current shrinkage of the law student applicant pool occurred. If the current admission practices of law schools is causing the bar passage decline, how do we explain that practitioners are doing no better?

Comments

[Michael Risch]

I agree with all you write here, but just want to note that the 75% bar passage standard is cumulative over time - 5 years or (as proposed) 2 years. So, you could average 68% first time, and then need only another 7% of your class (which would be a higher percentage of the actual takers, since fewer would be taking the exam) thereafter.

[William Patton]

Law School Transparency [LST] and their blog denizens have not focused on the real cause both of lower bar passage rates and the paucity of minority lawyers – unreasonable bar passage standards. State Supreme Courts have provided state bar associations with unbridled discretion to continue to ratchet up bar passage standards without demanding evidence-based data that doing so is either necessary or that it will result in greater consumer protection or better overall public policy. Look at California. The Supreme Court has permitted the State Bar to increase the passage score on the MPRE from 79 to 86 [national mean of 81], and the MBE cut score to 144 [national mean of 135] without any proof that those increases will either produce better or more ethical attorneys or that the public will be better protected. I recently provided the ABA with several empirical studies demonstrating that there is no proof that raising the bar score standards decreases state bar court ethics cases or reduces the number of legal malpractice or attorney incompetency determinations. The California State Bar has produced no data supporting their extremely high bar passage standards. However, those unproven standards have achieved one thing – a demographically unrepresentative bar. The United States Census states that the percentage of Hispanics in California increased from 32.4% in 2000 to 38.6% in 2014. However, according to the California State Bar, Hispanics comprised 3% of California's attorneys in 1999 and only 4.2% in 2011. It is time for LST and other groups to start lobbying State Supreme Courts and Bar Associations to produce data that their increasing bar passage standards are necessary in light of their effect on lawyer demographics. It is time for a reexamination of the competing public policies of consumer protection and increasing attorney racial and ethnic demographics rather than continuing the chorus about law school fraud.

[Kyle McEntee]

William, I happen to agree that some states do not have defensible bar passage standards. I simply do not believe that minimum competence is meaningfully different in California than in other states, which is what it would have to be in order to justify such a drastic difference in cut scores. I'm not sure where the right place to draw the line is. Neither am I saying a national standard is appropriate. But this is a problem that I routinely discuss with members of the press and at conferences, including at the National Conference of Bar Examiners annual conference. Advocating against unfair cut scores that serve only to protect the interests of the practicing bar are squarely within LST's zone of interest.

But this sidesteps the point of our criticism of law schools. I'm not sure if you're a golfer, but the best analogy is that we play the ball as it lies. Yes, the LSAT has racial disparities. Yes, the bar exam has racial disparities. We should address those problems without a doubt because our profession has an enormous diversity problem.

Yet the way to solve that diversity problem is not to allow schools to weaponize the cause of opportunity and access to keep tuition dollars flowing at the expense of students/graduates and better uses of those federal dollars. But that's exactly what several dozen schools are doing these days. We have to play it as it lies. We have a bar exam today, full stop. The bar exam has little to do with who will make a good lawyer, except in one major way: it's required to be a lawyer at all. While that's the case, and while law school costs as much money and time as it does, schools must be accountable for their outcomes. You're advocating for a lack of accountability because some groups across some states have created an unfair system. Reasonable or unreasonable, the barrier remains and ignoring it ruins lives of too many people who want to join our profession.

If you think we as an organization are a good fit for this work -- and I think we are -- help us find funding. We are still, after seven years, run on a shoe-string. I am the only full-time employee and my wages are astonishingly low by any comparison.

Or you can continue to lament outsiders like LST and call us names. Your call.

[Kyle McEntee]

Ralph, you make a pretty extraordinary claim here. You're essentially saying that the Florida Board of Bar Examiners is conspiring to keep people out of the profession. The Florida Bar has a protectionist reputation, but without evidence, it seems inappropriate to suggest that, despite not changing cut scores, they're somehow playing loose with the standards to keep people out.

[Ralph Clifford]

To Kyle McEntee 2:24:

No, I am not making that claim, but the arguments made here and elsewhere that all is the fault of money-grubbing law schools is likewise unsubstantiated; indeed, my main goal is to point out that comparing bar passage rates requires us to consider both the numerator (the possibility that the quality of applicants has changed) and the denominator (the possibility that the standards of success have changed). Either would change the pass rate.

[[M][a][c][K]]

I do not see where your data suggests that the pass rate for practitioners has declined. I should note that the Florida bar was considered even two decades ago to be hard on practitioners because of a desire to keep older lawyer-semi-retirees moving to Florida out of the local profession. Younger lawyers who are practitioners moving from one state to another may also be doing so because of a lack of success in practice wherevtey were before, something that might also align with a low bar passage rate.

Moreover, given the more comprehensive scope of the bar exam, for an existing lawyer to pass it may in fact be tougher than a new student. I had personal experience of this 5-6 into practice when I needed to sit a foreign bar QLTT - which was admittedly pretty extreme, I think it was 7 or 8 written 3 hour exam papers over a 5 days (this was in the 90s.) I passed, but found for 3-5 years after I'd get calls from other highly qualified practitioners also trying to pass later offerings (I'd recommend another country's QLTT which was a lot easier.)

[anon]

Ralph

Your observation that practitioners tens years post grad "do no better" than average on the bar demonstrates that you really don't have much of a handle on this issue.

After three years of law school, while the material is still fresh, nearly all law school grads fork out big bucks to take an intensive refresher/prep course for the bar. Demonstrate, please, that these practitioners, ten years or more post law school, are in a comparable position.

As for "opportunity" a post like William Patton's seems so confused. First, the participation by any group in the education system, and professional schools, is a complicated matter that has nothing whatsoever to do with the supposed discriminatory nature of the bar exam. Second, it is implicitly bigoted to claim that different standards for bar exam passage should be applied to different groups based on ethnicity or race. If Mr. Patton believes that the bar is a test for "whiteness" than he should say so.

As for supposed proof that there are reliable studies that show no correlation between civil liability for legal malpractice, discipline by state bar associations and slightly increased standards for bar passage, I would simply state that such proof cannot exist, so one would not expect to find it. Modest changes in "cut points" on bar passage is too attenuated to be deemed a causal factor affecting these post passage measures. Discipline is meted out to attorneys in practice for many years - too many to show an increase or decrease based on modest changes in bar pass "cut points" referenced. The fallacy in the Patton analysis is that standards on the bar exam are directly intended to directly influence rates of civil liability and discipline. Neither of these measure, truly, is directly related to competence in the abstract, especially the latter, which often has nothing to do with it.

Finally, Mr. Patton betrays his bias. The "tell" - his ranting against LST and "scambloggers" who are complaining about "law school fraud" - is so out of date. This is the soldier in the cave who doesn't realize that that war is over - and his side lost.

[William Patton]

Kyle,
Your golf analogy is a good one. LST supports the ABA proposed new Standard 316 that reduces the bar passage accreditation standard from 5 years to 2 years. You therefore are on record as supporting a national bar standard. Let us apply your golf analogy to your support for Standard 316. In golf your score [success of the round] is factored based upon which set of tees you use. Your “golf handicap” is determined in part by the length of the golf course. By contrast, under Standard 316 the rule applies to all law schools even though state bar associations have provided much different length courses and tee off areas [MBE cut scores]. For instance, this week in the National Law Journal Dean Boise, Syracuse, said he supports the proposed ABA standard. The problem is that his school has been judged by the New York MBE cut score of 130. But Syracuse graduates fail miserably when they take a difficult bar examination (California MBE cut score of 144). Ninety-two Syracuse graduates took the July California Bar Examination from 2010 to 2015. The Syracuse California first-time passage rate was an abysmal 27% (25 out of 92), and the Syracuse graduates’ repeater passage rate was only 24% (10 out of 41). A national standard that shields schools like Syracuse that are protected by lower state bar passage standards but punishes schools, like those in California, that face high bar passage standards that schools like Syracuse cannot meet, is obviously unfair and will only exacerbate the demographic imbalance in the bar. Those are some of the reasons why I oppose ABA standard 316.
If we agree that state bar standards must be data based, not just based on monopolistic goals, then I agree that we should work together and that LST should spend as much time trying to change those unreasonable standards as it does framing the issue of consumer student fraud. Let’s talk.

[Kyle McEntee]

My email address is kyle @ law school transparency dot com.

There's a difference between a national bar standard (i.e. a 135 cut score in every state) and a standard by which all accredited law schools must be measured against and held accountable to. I am on the record supporting the latter, but not the former.

This distinction matters because it shows why your extension of my analogy is misplaced. The exact same cohort will perform different based on the state. But the state a student chooses to take the bar exam in reveals an important preference. It's no use to that student to tell them, "oh you would have passed if only you'd taken the exam in another state."

Indeed, this holds schools to different standards of student quality and instruction. That's just too bad. It's a consequence of framing the issue as student-centric instead of school-centric.

[William Patton]

Anon,

You miss the point entirely. I am not talking about race based testing. What I am talking about is how extremely high bar standards like those in CA limit the number of minorities licensed to practice law in CA. For instance, on the July 2015 CA bar 118 Black students from CA ABA law schools passed the exam for a 53.4% passage rate. If CA had even the national median MBE cut score of 135 instead of 144, a much larger number would now be licensed to practice law. Do you really believe that those who passed the New York bar at a 130 MBE cut score, but who later take and fail the CA bar with a 144 MBE cut score should not be allowed to practice law in CA because they are not minimally competent? The reality is that unreasonably high bar passage standards limit access to the profession for everyone and thus necessarily reduce the number of minorities admitted.

[William Patton]

Anon,

You state that it is impossible to conduct a study to determine the correlation between passing a bar required examination and discipline by state bar associations. You are wrong. In a recent study I examined 3 years of Wisconsin published state bar disciplinary cases to determine whether having passed the MPRE had any effect on attorney ethics cases. In Wisconsin most attorneys are admitted under the diploma privilege, but out-of-state attorneys must pass the MPRE to be licensed. My analysis demonstrated that there was no statistical correlation between passing the MPRE and the rates or severity of attorney discipline. Those admitted under the diploma privilege without taking the MPRE and those who passed the MPRE had extremely similar disciplinary histories.

If there is no correlation between passing the MPRE and attorney conduct, then what justifies state bars' different MPRE passage scores which range from 75 [nine states] to 86 [CA and Utah].

My point is that unrealistically high and empirically unjustified bar standards limit access to the profession and frustrate other social justice goals.

[anon]

WP

Wow. YOu are doubling down, and I can see the reason. You have taken this position before.

However, the logic of your arguments seems deeply flawed.

First, you compare "diploma privilege" to "MPRE" passage. Without claiming Wiki to be a reliable source, therein, it is stated "In Wisconsin, graduates of the two American Bar Association-accredited law schools in the state (Marquette University Law School and the University of Wisconsin Law School) may seek admission to the State Bar of Wisconsin without having to sit for a bar examination."

So, your argument is: the rate of attorney discipline of Wisconsin law school grads, who don't take the MPRE, when compared with out of state attorneys who do, is the same, and therefore, the MPRE passage does nothing to ensure attorney ethics. The illogic here is stunning.

First, your "study" appears to be based on the argument (implicitly) that, although the Wisconsin law schools don't teach ethics, Wisconsin attorneys are just as ethical (or unethical) as out of state attorneys who take the MPRE. What nonsense!

Second, how irrelevant! Your whole line of argument on this point is patently irrelevant to a dispute about bar pass requirements. The "bar exam" tests, generally, doctrinal subject matter competence (and sometimes, some element of "skill"). The MPRE is a separate requirement. The bar exam is the matter under discussion. Your claims about the MPRE efficacy are just irrelevant.

Third, you seem to believe that differing "cut points" on different tests are inherently unfair and don't make any difference in the long run. If this is true, and it may well be, you haven't presented a shred of evidence to support that claim.

Finally, you double down on this confused mess by mixing your arguments about the efficacy of MPRE pass rate and efficacy with your "opportunity" claims. Again, you mix up your MPRE assertions with the "bar pass" requirement, and you ignore the deeply flawed nature of your discrimination claims with respect to the bar exam. See, above.

[Matthew Reid Krell]

William,

Where is this study available for people to read?

[David Frakt]

The cause for dropping bar passage rates, except in states where the state board of bar examiners has made a specific choice to raise the cut score, is the poorer aptitude of the cohort of students taking the bar examination. Reasonable people can differ about what should be tested on bar examinations, the best format for bar examinations and the appropriate cut score to demonstrate minimum competency. But law schools are required to make admission decision based on who has the aptitude to pass the bar exam that is currently being administered. Law schools know what state they are located in, and they know which state bars their students tend to take. They therefore have to calibrate their admissions to ensure that the students they admit can pass the bar in the state or states where they take the bar at the rate required by the ABA to maintain accreditation. Meanwhile, if they want to advocate for reforms in bar examination or the bar admissions process, they are free to do so.

[anon]

David
Well said.

ANd, a postscript with respect to the notion of an "opportunity" pass on the bar exam.

What would the advocates of such a system recommend for testing?

[Ralph Clifford]

A couple of responses:

1. Bar examinations can be made more difficult without changing the "cut" score, particularly the essay section. The questions can be made harder involving more issues or more difficult ones (an ERISA question should sort things out ...). Also, unlike the Multistate, grading the essays is fundamentally subjective. If the graders of the questions start giving threes rather than fours, the bar pass rate would decline. (And no, I'm not suggesting that there is a conspiracy to do this, but it could nevertheless happen).

2. I do know about moving to a new jurisdiction after ten years of practice as I did that.

3. Admitting a student to law school is always a guess. Even at their best, LSAT and UGPA are far less than completely reliable. Students with great predictors will tank while those at the "bottom" will do amazingly well.

Please understand that I do not have a "side" in the on-going debate. What I hope for is that the complicated questions involved are not over-simplified. The solutions created should solve the problem.

[Scott Fruehwald]

Professor Patton, Is your law school doing anything to better educate your students? It seems to me that if you want your students to pass the bar and become effective lawyers, your faculty would be devoting more time to teaching than the average law school.

[Twbb]

Ralph, Occam's razor would counsel against raising such hypothetical factors without any evidence. Especially since there is already an established correlation between LSAT performance (measuring literacy and critical thinking) and the Bar (measuring literacy and critical thinking). Also as a member of the Florida Bar who follows Bar news fairly closely I have not heard of even a rumor that exam difficulty has increased.

[William Patton]

Anon,

You keep incorrectly calling my arguments "discrimination claims". Let me see if you can follow this simple argument: (1) if a bar passage standard is unreasonably difficult (i.e. not based on empirical data that demonstrates its necessity or social value) the result will be that fewer of those who are otherwise sufficiently competent and who have spent tens of thousands of dollars to practice law will be licensed; (2) if fewer pass the bar then fewer men, women, racial and ethnic minorities will practice law; (3) there is no evidence that an unreasonably difficult bar standard better protects the public;(4)the ABA and almost every state bar association for the past thirty years has adopted a goal of increasing diversity within the bar; (5) if the goal of bar associations is to both reasonably protect consumers and increase diversity then an unreasonably difficult bar standard frustrates one goal and does nothing to increase the other. I do not understand why you call this a "discrimination claim." I am not arguing for an easy bar exam or that law schools should not be accountable for outcome measures.
Do you frame my argument as a "discrimination claim" because you are against increasing diversity within the legal profession, and if so, why?

[William Patton]

Scott Fruehwald,

Scott, although I taught at UCLA and Whittier Law Schools, I am retired, and no longer have any formal or informal relationship with any law school. However, I still have a serious concern about the impact of unreasonable national accreditation standards on diversity within the profession.