The relative difficulty of state Bar exams has popped up as a topic of recent interest.
- The Illinois Board of Admissions to the Bar announced that it will raise the minimum passing score, or cut score, on the Illinois Bar exam from 264 (out of 400) to 272. The cut score will be raised in two stages, first to 268 for the July 2013 Bar, and second, to 272 for the July 2015 Bar.
- Robert Anderson (Pepperdine) recently blogged on The Most Difficult Bar Exams, and his post was picked up by Above the Law and the ABA Journal News Now, among others.
So, how much harder is the Illinois Bar becoming?
Forty-five out of 51 jurisdictions (the 50 states, plus the District of Columbia) have adopted two key grading practices: scaling scores and using combined scores.
- Scaling helps ensure that differences in scores from component to component, or from administration to administration, are due to differences in performance, rather than to differences in the relative difficulty of questions, or of the graders. The NCBE uses questions from prior exams to scale the raw scores on the MBE. Forty-six states use the distribution of the final MBE scores in their states to scale raw scores on the state components of their Bar exams.
- Forty-five of the scaling states combine the scaled scores on each of the components, and then compare the combined score to the state's minimum passing score.
The wide adoption of both scaling and combined-score grading means that the primary determinant of the relative difficulty of a particular state's Bar exam is the state's minimum passing score, or cut score. Chart 9 of the Comprehensive Guide shows the cut scores for most states. First, Chart 9 shows the cut score on the scale used by the state (e.g., 268 out of 400). Because the states use different scales, Chart 9 also converts each state's cut score to a common scale in which 200 points is the highest score. The remainder of this piece will use the 200-point scale in referring to state cut scores.
For the July 2013 Bars, state cut scores will vary from a low of 128 (Alabama) to a high of 145 (Delaware), which just edged out California's 144. The average cut score is 135.1. The most common cut score is 135, which will be used by 15 states. The following chart shows the distribution of the July 2013 Bar cut scores:
The following table ranks the State cut scores from highest to lowest.
| July 2013 State Bar Exam Cut Scores |
||
|---|---|---|
| State | Cut Score | Rank |
| Delaware | 145.0 | 1 |
| California | 144.0 | 2 |
| Oregon | 142.0 | 3 |
| Alaska | 140.0 | 4 |
| Idaho | 140.0 | 4 |
| Nevada | 140.0 | 4 |
| Virginia | 140.0 | 4 |
| North Carolina | 138.4 | 8 |
| Colorado | 138.0 | 9 |
| Maine | 138.0 | 9 |
| Rhode Island | 138.0 | 9 |
| Arizona | 136.5 | 12 |
| Florida | 136.0 | 13 |
| Pennsylvania | 136.0 | 13 |
| Maryland | 135.3 | 15 |
| Arkansas | 135.0 | 16 |
| Georgia | 135.0 | 16 |
| Massachusetts | 135.0 | 16 |
| Michigan | 135.0 | 16 |
| Montana | 135.0 | 16 |
| Nebraska | 135.0 | 16 |
| New Hampshire | 135.0 | 16 |
| Ohio | 135.0 | 16 |
| Oklahoma | 135.0 | 16 |
| Tennessee | 135.0 | 16 |
| Texas | 135.0 | 16 |
| Utah | 135.0 | 16 |
| Washington | 135.0 | 16 |
| West Virginia | 135.0 | 16 |
| Wyoming | 135.0 | 16 |
| Hawaii | 134.0 | 31 |
| Illinois | 134.0 | 31 |
| District of Columbia | 133.0 | 33 |
| Iowa | 133.0 | 33 |
| Kansas | 133.0 | 33 |
| New Jersey | 133.0 | 33 |
| New York | 133.0 | 33 |
| Connecticut | 132.0 | 38 |
| Indiana | 132.0 | 38 |
| Mississippi | 132.0 | 38 |
| Minnesota | 130.0 | 41 |
| Missouri | 130.0 | 41 |
| New Mexico | 130.0 | 41 |
| North Dakota | 130.0 | 41 |
| South Dakota | 130.0 | 41 |
| Wisconsin | 129.0 | 46 |
| Alabama | 128.0 | 47 |
By raising its cut score from a 132 to 134, Illinois moved from a tie for the 37th hardest Bar exam to a tie for the 31st hardest. If Illinois had moved directly to a 136 cut score, it would have tied for 13th hardest.
Rank is one thing, but what is the likely effect of cut scores on examinees? To look at this question, I took data from the NCBE's 2010 Bar statistics table "2010 First-Time Exam Takers and Repeaters from ABA-Approved Law Schools"). For each state, I took the number of ABA first-takers from ABA who took the Bar exam in that state, as well as the number who passed. I then added the cut score of each state, taken from the Comprehensive Guide for 2010.
As I discussed in Bar Exam Cut Scores vs. ABA First-Time Bar Passage Rates, the ABA Bar passage rates generally fall as cut score rises. That said, the ABA passage rates of the states vary widely, even after cut score is taken into account. As Prof. Robert Anderson (Pepperdine) discussed in The Most Difficult Bar Exams, much of that variation may be due to differences in the average LSAT scores of takers in a particular state. That would be consistent with my own analyis of law-school Bar passage rates. In Unpacking the Bar: Of Cut Scores and Competence, 32 J. Legal Prof. 67 (2008), I concluded that both the LSAT scores of entering classes and cut score affected the Bar passage rates of law schools.
To try to offset the effect of state-specific factors, such as the average LSAT of ABA first-takers, I aggregated the state data by cut score. For each cut score used by one of the states, I determined the total number of takers, and of passers, in states that used that cut score. I then calculated the Bar passage rates for each cut score.
As the following chart shows, the relationship between cut score and Bar passage rates of ABA law school examinees emerges even more strongly:
The most obvious outliers are (128, 84.9%, and (134, 77.0%). Each includes the 2010 results of only a single state (128, Alabama; and 134 Hawaii), and thus are more likely to have been affected by state-specifc factors. If nothing else, the data points include relatively few takers: Alabama, 444 takers; and Hawaii, 196 takers.
The fit line comes from a weighted regression of Bar Passage Rate against Cut Score. The coefficient for Cut Score was -0.012, which means that the expected Bar Passage Rate drops by 1.2% for every one-point increase in Cut Score.
Back to Illinois and its cut-score increases. The two-point increase for July 2013 means about a 2.4% drop in the expected Bar passage rates of ABA law-school examinees. The July 2015 2-point increase would lower expected Bar passage rates another 2.4%, for a total decrease of about 4.8%.
One hopes that the reason for the two-year delay before the second cut-score increase is to give the Illinois Board of Admissions to the Bar the time to do what the New York Board of Law Examiners did. In New York, the BOLE had planned to raise the cut score by 3 points, over three years, in one-point increments. The BOLE commissioned a study of the effects of the initial cut-score increase, especially on minorities. After seeing the results of the study, further increases in cut score were abandoned.
Note: you can find the several reports commissioned by the New York State Board of Law Examiners on the NYS Bar Exam Reports and Press Releases page on their website.
Gary Rosin
Comments