Bar Exam Cut Scores vs. ABA First-Time Bar Passage Rates

As I discussed in Interpretation 301-6:  Low LSATs and High "Cut Scores", both law-school LSAT scores and state cut scores affect law school Bar passage rates.  The following chart shows the 2010 overall first-time Bar passage rate for takers from ABA-approved law schools in each state, by its cut score.

Sources: Comprehensive Guide to Bar Admission Requirements (2010) and National Conference of Bar Examiners, 2010 Statistics, 2010 First-Time Exam Takers and Repeaters from ABA-Approved Law Schools (Bar Examiner, March 2011, pp. 16-19).

While the Bar passage rates vary widely, even among states with the same cut score, passage rates trended lower as cut score increased.  The curved fit line is from a logistic regression of Bar passage against cut scores.  The model accounted for about 68% of the deviance in Bar passage.  In general, a one-point increase in cut score decreased the odds of a graduate of a law school passing the Bar by 7.6%.

The variation in Bar passage rates among states with the same cut score could be due to numerous factors, including the distribution of LSAT scores of ABA first-takers in that state.  For cut scores used by four or more states, the chart show the ste (jurisdiction) with the lowest and the highest passage rate.  California is noted separately, because it has the second-highest cut score and the second-highest number of ABA first-takers (5902).  New York (133, 84.9%) had the highest number of ABA first-takers (9092).

Comments

[Matthew Reid Krell]

Given the closeness of fit for New York and California, I wonder if you controlled for each state's size effect.

[Matthew Reid Krell]

And this one I'm less sure of, but I think that you're supposed to use a linear regression for ratio-interval DVs.

[Gary]

I used a Generalized Linear Model with a logit link function. I treated the data as a number of events (passes) occurring in a set of trials (takers)and used the binomial distribution.

The resulting "curve" is very close to linear. In this case, the correlation coefficient for (cut score, passage rate) is slightly higher the coefficient for (cut score, logit(passage rate))(-.603 vs. -0.568, both significant at or above the 1% level).