The New York Bar Examiners released the results of the July 2017 bar results by law school earlier this week. Overall the first-time bar pass rate for ABA schools was up (86% compared to 83% last year), and most New York law schools saw a modest increase in their results. But, as might be expected in a state with 15 ABA accredited law schools, there were some outliers. These high and low performing schools were featured in an article by Staci Zaretsky at Above the Law. Staci noted that a couple of schools saw dramatic increases while a couple of others saw significant decreases in their first-time pass rate. She sought comments from the schools to explain why these results. In this post, I examine possible explanations for the schools with the strongest and weakest performances.
The ATL article reports that Albany Law School's first-time bar passage rate in New York dropped from 82.7% in July 2016 to 71.3% in July 2017. Whenever I see a law school's bar exam results go down, the first thing I do is go back and look at the admissions statistics from three years earlier, then compare them with the prior year. More often than not, a significant drop in bar passage rate can be explained by a corresponding decline in admissions standards. Not surprisingly, when we look at the incoming credentials of students Albany admitted in 2014 compared to 2013, we see a sharp decline. In 2013, the 75/50/25 LSAT scores at Albany were 156/153/150, meaning that, at most, 24% were at 149 or below, the point at which law students start to be at high risk of failing the bar. (See LSAT risk bands chart below) In 2014, the numbers declined across the board to 154/151/148, suggesting that somewhere between 30-40% were at 149 or lower, and therefore in a high risk of failure category. Undergraduate GPAs also declined across the board from 3.59/3.38/3.13 in 2013 to 3.51/3.22/2.90 in 2014. This significant drop in the capability of the admitted students easily explains the drop in bar passage rates three years later.
New York Law School also experienced a fairly significant drop from 70.3% to 64.7%. This drop can also be explained by a change in admission standards. An analysis of NYLS admissions statistics reveals a dip in the admissions credentials from 2013 to 2014 at the bottom of the class. In 2013 NYLS numbers were 153/151/149 and 3.43/3.17/2.87. In 2014, they were 154/151/148 3.40/3.20/2.80, the key factor being the drop at the 25% from 149/2.87 to 148/2.80 Because about a quarter of NYLS students are enrolled part-time, it is also helpful to compare the credentials of 2013 part-time admits with 2012 admits. The part-time group declined from 153/150/149 in 2012 to 152/150/148 in 2013 with a slight drop in GPAs as well. Again, this increase in the admissions of high risk students three and four years earlier can explain the decline in the bar pass rate from 2016 to 2017.
The third New York school that experienced a decline in bar passage this summer was Brooklyn, from 82.7% to 78.6%. Brooklyn sharply lowered their admission standards between 2013 to 2014. For full-time students, the bottom 25% went from 158/3.16 in 2013 to 153/3.05. This is a statistically huge drop. (Brooklyn's part-time entering class credentials also declined slightly at the 25% percentile between 2012 and 2013, although part-time students make up only 12-15% of the class.) According to my LSAT risk bands, students with LSATs above 156 are at minimal risk of failing the bar, while those at 153-5 are still at low risk. But students at 150-152, a group I call "modest risk" are definitely at a heightened risk of failure. When Brooklyn went from having few, if any, at-risk students to up to 24% of the class at modest risk or worse, a decline in the bar pass rate was not only predictable, it was virtually inevitable.
In contrast, St. John's, which had the biggest increase in the first time pass rate (from 76.5% to 89.2%) actually raised their admission standards from 2013 to 2014 across the board from 159/156/153 and 3.66/3.41/3.17 to 160/157/154 and 3.65/3.48/3.22 for their full-time students, even though that meant shrinking the incoming class from 216 to 180, a 16% drop. This wise decision to admit a smaller but stronger class, perhaps along with increased emphasis on bar preparation at St. John's, paid dividends with the increased bar passage rate this year.
One school that deserves great credit for its increased bar passage rate is CUNY, which raised its rate from 75% to 86.8% (above the state average) despite lowering their entrance credentials at the bottom 25% from 2013 to 2014. In addition to improvements made in their bar preparation program, it appears that CUNY also did an improved job at holding down transfer attrition from the top of the class, which also likely contributed to their increased rate. I am concerned, however, about CUNY's ability to maintain its strong bar performance next year and beyond. CUNY substantially lowered its standards from 2014 to 2015, and again in 2016. The school also started a part-time program in 2015, and the students in the part-time program have even weaker credentials. Those students will start to take the bar next year.
Law schools can make changes to their curriculum, their academic support programs, and their bar preparation programs that will have a positive impact on bar passage rate, but the single most significant factor in bar passage rates remains the caliber of the admitted students. That is why NYU, Columbia and Cornell all had first-time pass rates above 95%, with Fordham close behind at 93.4%. These schools admit virtually no at-risk students. The caliber of admitted students also explains why Touro fared quite poorly on the bar (64.8%, just edging out NYLS by .1%). Touro had the weakest entering class of any New York law school in 2014 at 149/146/145. Hofstra, with a 2014 LSAT 25% at 147 (the 33rd percentile), also struggled (70.4%). Even in a state with a relatively low cut score like New York, a high percentage of graduates will still fail the bar if they are in the bottom 1/3 of LSAT takers. College graduates contemplating law school who are unable to score above 147 should strongly consider other career options.
David Frakt's LSAT Score Risk Bands (Risk refers to likelihood of failing out of law school or failing the bar)
156-180 Minimal Risk
153-155 Low Risk
150-152 Modest Risk
147-149 High Risk
145-146 Very High Risk
120-144 Extreme Risk
A one- or two-point change on the LSAT percentile of a school is not statistically significant. As the error band reported by the LSAC is 2.6 points, your statistical argument for both Albany and NYLS doing more poorly or St. Johns doing better is not sustainable.
Posted by: Ralph Clifford | November 30, 2017 at 09:51 AM
I beg to differ Ralph. A one or two point difference for an individual may not be statistically significant, because some students will be slightly better and some slightly stronger than their score. But when the whole class (or the bottom half of the class, where most of the failures come from) is weaker or stronger (and not just on LSATs, but also UGPA), then lower or higher performance can be predicted. Admission credentials may not account for all of the difference in bar performance, but it is certainly a significant factor.
Posted by: David Frakt | November 30, 2017 at 10:02 AM
I have not been following your posts regularly but you seem to assume there is a 100% correlation between LSAT score and bar passage. As I am sure you are aware, LSAT scores are designed to predict first-year grades not bar passage, and if you were right, it might just be possible to skip the whole law school thing. My sense is law school grades are quite a good predictor of bar passage, but the correlation between LSAT and law school grades (without factoring in UGPA) is not particularly strong. I share the concern that law schools are broadening their admission standards primarily out of financial rather than academic interests, but I just wanted to suggest you avoid overstating the connection between LSATS and bar passage.
Posted by: MLS | December 01, 2017 at 01:11 PM
I would encourage you to go back and read all my posts, as I have addressed this topic several times in previous posts. Your "sense" is incorrect. The correlation between LSAT and bar passage is very strong for law school classes as a whole, and even stronger with UGPA factored in, which is why I look to see if schools are balancing lower LSAT scores with higher UGPAs. (They seldom do.) Although first year grades are a better predictor of bar passage rates than LSAT scores, LSAT scores are by far the best predictor of bar passage that are available to law schools at the point of reviewing applications for admission.
Although the report is somewhat dated (1998), the LSAC National Longitudinal Bar Pass Study has some very useful information about the correlation between bar passage and LSAT scores. You can find the study here: http://www.unc.edu/edp/pdf/NLBPS.pdf
See also this more recent article from the NCBE which discusses the correlation of LSAT scores and MBE scores., and includes a table with average MBE score by LSAT score.
http://www.ncbex.org/assets/media_files/Bar-Examiner/articles/2011/800411Testing.pdf
I would also recommend Rosin, Gary S., Unpacking the Bar: Of Cut Scores, Competence and Crucibles. 1st Annual Conference on Empirical Legal Studies Paper; Journal of the Legal Profession, Vol. 32, 2008. Available at SSRN: http://ssrn.com/abstract=988429
For an analysis of eventual bar pass rates by LSAT score, see here:
http://www.thefacultylounge.org/2015/12/lsat-scores-and-eventual-bar-passage-rates.html
Posted by: David Frakt | December 01, 2017 at 01:55 PM