As I discussed in an earlier post, to understand bar passage problems requires more than a consideration of the preparation of the applicants for admission. If the exam becomes more difficult or its grading becomes stricter, the bar passage rate will decrease even if the quality of the applicants stays the same. Even David Frakt acknowledges this. See Some Thoughts on the California Bar Exam. Between the two of us, we have raised concern that both Florida and California have done exactly that—made the bar examination more difficult.
The purpose of this post is to add at least one additional state to this list: Colorado. Professor Scott Johns has posted an article on SSRN that compares the results students from his school, the University of Denver, have seen on the Colorado bar exam over the last years. See Scott Johns, Testing the Testers: The National Conference of Bar Examiners' LSAT Claim and a Roller Coaster Bar Exam Ride. After a particularly strong presentation about the statistical methodology he used in his study, he showed that the results on the Colorado bar demonstrate that it has become harder. The basic fact established in his study is that the students entering the University of Denver have become stronger on the objective measurements (particularly the LSAT) over the last years, but their bar passage rates have nevertheless declined. As he states:
As a law school that serves in a U[niform] B[ar] E[xamination] jurisdiction, we have a unique opportunity to evaluate the N[ational] C[onfrence of] B[ar] E[xaminers]’s primary claim that recent lower first-time bar passage rates are the consequence of declines in LSAT scores in contrast to more successful bar passage years. Based on empirical evidence, we do not find statistical support for the NCBE’s claim with respect to LSAT scores because those scores have remained relatively consistent (or slightly better) since 2008 (and since advent of the UBE in Colorado in 2012), and, yet, first-time bar passage rates have fallen for our law school. Simply put, the NCBE’s claim is unfounded, based on our empirical evidence.
Id. at 1.
The fact that Colorado is a Uniform Bar Examination state is significant. It is not an unfair assumption that the declining success among applicants taking the UBE around the country (now, half of the states plus D.C. use the UBE) has a source that is in addition to changes in the student population being admitted to U.S. law schools. Some of the declining percentage of success on bar exams must be attributed to the denominator changing as bar exams have become more difficult to pass.