Improving Bar Passage Rates for At-risk Students: Learning from Western State

It may well be that there are too many law schools in the country.  Clearly, the number of law school graduates in recent years has exceeded the number of jobs for new lawyers.  But it is equally clear that few law schools are likely to close down voluntarily.  No university administration wants to shutter an institution that employs dozens of professionals and staff, has a loyal alumni base and has a profitable history.  Rather, law schools will seek to endure the cyclical drop in applications through some combination of lower admission standards, reduction of class size, shrinkage of faculty and the institution of other cost-cutting measures until the legal job market improves and applications rebound. 

The weakness of the legal employment market and the skyrocketing cost of legal education has led to an historic decline in law school applications.  This reduction of the applicant pool has put the greatest pressure on what I will call “opportunity” schools – law schools which, by virtue of their mission or the market niche they occupy, traditionally admit a substantial number of at-risk students.  For the purposes of this essay, I am defining at-risk students as those at the 40^th percentile (149) or lower on the LSAT, although students slightly above that are still at some risk of failure (See the chart in my previous post.)  With law schools above them in the pecking order admitting students who, in more prosperous times, might have had no choice but to attend an opportunity school, these schools may be forced to admit an even higher percentage of marginal applicants in order to stay in business.  Of course, schools can go too far in lowering admission standards (see my discussion of InfiLaw’s recent admission practices in my previous post), but I believe it is possible for a law school to admit a substantial number of at-risk students without running afoul of ABA standards or ethical business practices. 

Whatever one’s view on the appropriateness of admitting at-risk students, I think we should all be able to agree that a law school, having admitted such students, owes them a duty to provide the greatest possible chance of success in realizing the

ir ambitions to earn a J.D. and to become a member of the bar.  This post addresses one way in which law schools may maximize the chances of success for these at-risk students.

My views on this topic largely derive from my experience at Western State Law School, where I was on the faculty from 2005-10, except for a one year military leave of absence.  During this time, Western State’s bar passage rate for first-time takers on the California bar went from a low of 28% to above 80%, a remarkable achievement that helped Western State earn full ABA accreditation.  (Note- Western State has sustained this high pass rate since I left).  What made this all the more remarkable is that these outstanding results were achieved by students with virtually identical entrance credentials to those who had done so poorly just a few years before.  During this time, a substantial majority of Western State’s students were in the at-risk category as I have defined it.  As a law school that had temporarily lost accreditation in 2004, then regained provisional accreditation status in 2005, and operated in the highly competitive law school market of Southern California, Western State was not able to simply raise admission standards.  Rather, we had to find a way to improve the performance of students with relatively modest entrance credentials.   In order to determine how to do this, we spent a great deal of time compiling and analyzing data concerning which students were likely to succeed and which had a high probability of initial bar failure.  It will not come as a surprise that students with a high GPA were most likely to pass the bar exam.  At Western State,  students with a 2.8 cumulative GPA at graduation, roughly the top 25-30% of the class, passed the bar at a very high rate, while students at 2.3 or below, the bottom 20-25%, rarely  passed the first time.  With students in the broad middle of the class, 2.4 to 2.7, success was problematic.  We needed to determine why some of these average students passed, while others with similar GPAs failed.  To summarize what we found, the key to bar passage was this:  sustained effort throughout law school resulting in consistent, solid performance in core/bar-tested subjects.

It is often noted that there is a very strong correlation between first year grades and bar passage.  While we certainly found this to be true at Western State, there were several instances where students who did quite well the first year failed the bar.  What we found is that many students who felt comfortable that they were doing well after the first year reduced their efforts somewhat in their second and third-year.  Other than highly motivated students who were seeking honors at graduation or competing for clerkships or positions with more prestigious firms or agencies, many students were content to lessen their academic efforts once they felt they were not in any risk of failure.  Students had a tendency to focus their energy on activities that were more fun, like moot court or trial team or student organizations, or that they felt had more practical or job-related value like internships, externships or part-time employment with law firms.  There was also a tendency for students to put the greater weight of their academic effort into non bar-related elective courses, perhaps because they considered the subject matter more interesting or more potentially relevant to their future practice (e.g. trial advocacy).

To counteract these tendencies, we instituted a requirement that we called Foundation Law Points (FLP).  This program required students to earn a grade of 2.5 or better in a minimum of 8 courses from a list of about 25 bar-tested or fundamental courses.  At least 4 FLPs had to be earned in the first year, and, in order to ensure sustained effort in the upper division, students had to earn 4 more FLPs in their second or third year (you can read the specifics of the program in the current student handbook found here at section III:5)

The idea behind FLPs was a simple one.  When you think about it, the bar exam is basically a giant law school exam.  Depending on where the state bar examiners set the cut line for passage, what is required to pass the bar is typically equivalent to between a C+ and B- level of performance in law school.  Thus, a student who can consistently perform at a 2.5 level throughout law school in bar-tested courses should be able to pass the bar.  The FLP program gave students the incentive to work up to their full capacity in the classes that mattered most for bar passage.  It gave them an understanding of what level of effort was required to succeed in bar related subjects and most importantly, it provided them with a sense of confidence and even pride when they realized that they were capable of performing at a strong level in demanding courses. I should note that during the implementation phase of this program, which we applied not only to prospective students but to current students on a pro-rata basis, each at-risk student was counseled about their performance and work habits and was given a clear statistical understanding of what bar results were likely if they did not improve their academic efforts.  Failure to meet the FLP requirements resulted in being placed on probation, just as if the student dropped below the required cumulative GPA.  One unique aspect of the program was that we permitted students who otherwise had completed graduation requirements but lacked the necessary FLPs to retake FLP courses at no cost.

Although Western State instituted a wide variety of programs designed to increase the bar passage rate, I believe the Foundation Law Points program was the single most effective one.  Law schools with similar student academic profiles should consider such a program.  While there may be other ways to achieve the same effect, it should be noted that simply raising the cumulative GPA requirement to be in good standing or to graduate would not necessarily do so.  In fact, raising the cumulative GPA requirement might simply encourage students to avoid what they considered to be the toughest classes and focus their efforts on those elective classes where they felt they had the best chance to get a good grade.  It also should be emphasized that a program like FLP will not work unless there are clear grading guidelines in effect that are calibrated to the quality of the students being admitted.   It also is beneficial if faculty members have a sense of what level of performance is required to pass the bar in their state.  In California, the state bar examiners allowed law professors to sit on bar grading calibration sessions, and I found it very instructive to do so.  It is worth checking with your state bar examiners to see if they have a similar program. If not, encourage them to start one. 

In my next post, I will offer some additional recommendations.  I welcome your comments.

 

 

Comments

[Michael Risch]

Yes, I know that first time, ABA approved bar passage is higher in CA. And yet, CA allows non-ABA schools so long as 1L's pass the baby bar. Maybe all the ire should be directed there? We do have standards, after all.

More particularly, the numbers above say 30% attrition, not 50% attrition. And the standards are that students must appear capable of graduating and passing the bar. Let's say it's half, but we can't tell which half. What I'm saying is that we should figure out a way to give half a chance, preferably cheaply. What you're saying is that the half that might succeed should be denied the opportunity, ostensibly to save the half that won't, even though 100% are willing to take the risk.

[anon]

My, it's tough to get thru.

Here are the stats from a comment above, no reason to doubt them:

"Some data points on Western State's recent student attrition rates.

From lawschoolnumbers.com's stats on attrition rates:
1st year: 27.5%
2nd year: 13.7%
3rd year: 3.1%

From the school's 2013 509 Report (J.D. Attrition):
1st year: 28.1%
2nd year: 7.2%
3rd year: 2.5%"

We are scolded by MR that "the numbers [for WS] show 30% attrition." No, actually that isn't true.

Moreover, no one said WS was at the 50% standard (yet) that MR made up out of whole cloth. It was noted that MR's standard exceeded WS's actual attrition rate, but not by much.

Just like the CA bar pass rates, and the ABA standards, we are arguing with someone's imagination, rather than reality and common sense. Common sense dictates that all standards do not need to go away because someone barred by such standards might ultimately achieve.

Argue for open admissions? Fine. But MR doesn't do that. So, MR's point is hopelessly confused.

Implicit in MR's comments is that everyone has a right to attend law school, and expressly stated in his commentary is the assertion that, so long as the abysmal facts are disclosed, law schools with attrition rates of about 40% (but not 50%) should proliferate, so that some unspecified number of applicants may have the opportunity to someday pass the bar after taking a three year prep course.

Sorry. That doesn't sound like a good idea.

[Michael Risch]

Yes, we are on the same page, anon. My numbers may be imaginary, but might I remind that the comments were busting Prof. Frakt's chops about Western State, with a 150 median, well above what I'm talking about. And so I wonder at what point it IS a good idea to let people try to become lawyers? This prompted my elitism comment.

But let's focus on my point about the lower end and my admittedly arbitrary 50%. The mean score for African Americans is around 142, for hispanics around 147. So, the upshot is that your "not a very good idea" - in the name of protecting these people of color - instead locks out people of color by not allowing them to try to get a legal education, even if half of them might succeed and become lawyers.

This is our fundamental divide. Should they pay $60K for that opportunity? No, we should look for better, faster, and cheaper ways to sort the top 50% from the bottom 50%.

Will they pay $60K? Unlikely, but even if they do, the question is - assuming that these facts are all known and disclosed (a tall order, I grant you) - what right you, or even the ABA, has to decide whether $60K is worth the shot at such a career. The ABA has standards, and my assertion is that 50% could satisfy the standard if folks aren't pushed through to graduation with no shot of passing the bar.

[JillyFromPhilly]

Professor Risch,

Is there anything stopping people scoring in the 140s to study harder and raise their score. Wouldn't requiring a 150+ LSAT score cause less colateral damage than enrolling hundreds of students that will flunk out or fail to ever work as lawyers upon graduation, leaving them burdened with tens or hudreds of thousands of dollars of debt, with nothing tangible to show for it?

On your accusations of elitism: Law is an incredibly competitive profession. If you are not among the relative elite, or you do not have some kind of close connections to the business world that would allow you to drum up business, you are going to have an extremely difficult time making a living as a lawyer in today's economic climate. 4th tier schools are setting up the 80% of their classes that finish outside the top 15% and do not have business connections for failure. The opportunity they are offering the vast majority of their classes is life ruining debt.

When you say, "will you they pay $60k? Unlikely," what are you basing that comment on? My understanding of law school economics is the poorer LSAT scorers in any given class are almost always the ones paying full frieght? Do you have any data that suggests otherwise?

[David Frakt]

JillyfromPhilly -

In my experience, most students scoring in their 140s have already spent a great deal of time and often a considerable amount of money prepping for the LSAT. Knowing how much is riding on their exam score, students with a history of poor performance on standardized tests often take multiple LSAT prep courses and/or engage in extensive self-study. Also, they often take the LSAT multiple times. Interestingly, while many lower-tier law schools used to average the reported LSAT scores (which provides a more accurate picture of the students aptitude) many now only consider the highest reported score. While intensive prep for the LSAT may result in a marginal score improvement, it typically would not boost a student from 144 or below into the 150s. Even if students could somehow magically raise their LSAT score through study, the higher score would not actually reflect greater aptitude for the study of law. Rather, it would reflect mastery of the particular format of the LSAT and effective standardized test-taking strategy. These skills will be of little value in law school. So, no, this approach would not "cause less collateral damage" to borrow your phrase. It is important to bear in mind that with over 30 schools with a median LSAT below 150, we are talking about thousands of students that would have to substantially raise their scores if law schools were going to stop admitting students with scores in the 140s. This is exceedingly unlikely to happen.

NOTE TO ALL COMMENTERS - While the debate about admitting at-risk students is interesting and important, for the foreseeable future, it appears certain that a significant number of schools will continue to admit a substantial number of such students. Therefore, I would prefer to focus the discussion on what law faculties can do to help these students succeed. Forcing these students to put added effort into the core curriculum and to maintain that effort through graduation has proven to be effective at Western State in raising the bar passage rate. I would welcome contributions from others who have other constructive ideas on the topic of raising performance of at-risk students. Since we seem to just be going in circles, I am going to start removing comments that simply rehash the "law school scam" debate.

[John Steele]

Here's the article about what NYLS did to improve bar passage.

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1551477

[David Frakt]

Thank you, John

[anon2]

David -- the Atlantic article mentions that you "noted that according to statistics from the Law School Admission Council...scores higher than those in the 60th percentile correlate with a low risk of failing to eventually pass a bar" while "scores below the 25th percentile correlate with an extreme risk of failure."

Is there a specific LSAC report which you were referring to in your presentation? I'm interested in finding out more but am not familiar with the LSAC research referenced.

[David Frakt]

There is a tremendous amount of research on all things related to the LSAT on the LSAC research page http://www.lsac.org/lsacresources/research/all/tr

I urge those who are honestly trying to understand LSATs as an admissions tool to review this research.

There is a lot of research at this page on how well the LSATs correlate with law school success (very strong correlation with performance in the first year of law school. The LSAT alone is more predictive than UGPA alone, but combining the two together provides even more accurate predictions. ) There are reports on what students do to prepare for the LSAT (average students uses 2-3 different study methods). There are reports on the performance of repeat test takers. (Average score goes up 2.8 the second time you take it, another 2.2 points the third time.) There is a report on Analysis of Differential Prediction of Law School Performance by Racial/Ethnic Subgroups which convincingly demonstrates that the LSAT does not underpredict the performance of the minority groups studied. There is a report on the validity of LSAT scores for repeat test takers which indicates that using the average score is a better predictor than using the highest score.

Although the report is somewhat dated (1998), the LSAC National Longitudinal Bar Pass Study has some very useful information about the correlation between bar passage and LSAT scores. You can find the study here: http://www.unc.edu/edp/pdf/NLBPS.pdf

See also this more recent article from the NCBE which discusses the correlation of LSAT scores and MBE scores., and includes a table with average MBE score by LSAT score.
http://www.ncbex.org/assets/media_files/Bar-Examiner/articles/2011/800411Testing.pdf

I would also recommend Rosin, Gary S., Unpacking the Bar: Of Cut Scores, Competence and Crucibles. 1st Annual Conference on Empirical Legal Studies Paper; Journal of the Legal Profession, Vol. 32, 2008. Available at SSRN: http://ssrn.com/abstract=988429

All of this research as well as my own research of several years of student data at Western State informed my presentation at Florida Coastal.

[anon2]

Much appreciated David for those lists of resources. I'll definitely check them out. Is there a published analysis anywhere (by LSAC or others) which shows that an LSAT of <145 has x% chance of passing the bar, 145-149 a y% ... etc, etc. or was that your own analysis by triangulating different studies or based on private sources of information?

I'm familiar with the 1998 LSAC study but it seemed to me that the LSAT scores were on a different scale back then (on the bottom of page 24 it notes: "the LSAT data are shown on the LSAT scale, which had a mean of 36.8 and a standard deviation of 5.5").

Much appreciated

[David Frakt]

I never attempted to offer a percentage chance for passing for those below 145. I don't believe the publicly available data is sufficiently granular for that. Furthermore, the chances of passing would depend in significant part on which state bar one was taking. All I said was that students with a 144 LSAT and below, on average, were at what I considered to be an extremely high risk of either failing out of law school or failing the bar. Florida Coastal recently issued a press release indicating that their graduates who entered law school with an LSAT score below a 145 had a 63% first time bar pass rate. The press release did not include any data on the academic attrition for those with LSATs below 145, nor did it provide any information about the sample size of the group of the years covered. Looking at the admissions numbers for entering students at Florida Coastal, it appears that relatively few students with LSATs below 145 were admitted before 2011, and the bar passage data for these students is not yet available.) My guess is that before Florida Coastal started to admit large numbers of students with LSATs below 145, they were fairly selective about which students they would take with such low LSAT scores. For example, they might have only taken students below 145 if they had a relatively high UGPA, suggesting reasonably aptitude for the study of law. Given that over half of the class now has a score below 145 (their reported median was 144), Florida Coastal clearly is no longer being as selective. I believe that any school with a median LSAT below 145 is going to have a very difficult time maintaining a bar pass rate that meets ABA standards, but only time will tell.

[anon2]

I'm not sure what happened but my previous post seems to have been deleted...

I mentioned that I have a close friend who scored a 141 or 142 on her LSAT and had been accepted to Florida Coastal and a few other places. She's incredibly hard working and I don't think her LSAT is a fair indication of her chance of being a great attorney.

However, we came across the Campos article (and several other articles) and we are now extremely concerned as it basically says below a 145 you have almost no chance of passing the bar. Professor David Frakt is the reference for that and the article says that Professor Frakt referenced in LSAC study as his source.

I'm just trying to figure out what the "data" is that supports this claim. The most I could find in the links Professor Frakt provided above is that LSAT has a strong correlation. The LSAC study seems to be from a time when the LSAT had a different scale.

Professor Frakt -- in terms of quantitative conclusions (>50% chance of passing vs. "quite unlikely" / well below 50% chance) does that come from your experience / Western State data primarily or is there somethings specifically across a broader set of schools that LSAC or somebody else may have published (or not published)?

I'd appreciate it if this post wasn't deleted unless I've inadvertantly violated some rule here...

[David Frakt]

Anon2. Although the LSAC study came from a time when LSATs were scored on a 12-48 point scale as opposed to 120-180 as it is now, you can still make comparisons based on the percentile score.

It would be irresponsible to predict an outcome for an individual based solely on one data point. When I was speaking at Florida Coastal, I was generalizing about a large cohort of admitted students. I do not claim to be the authority on LSAT scores. I was just offering my opinion based on several years of teaching at opportunity schools and the review of the sources noted above. If your close friend is incredibly hard working that is a useful attribute that will stand her in good stead in law school. If she was a very good undergraduate student with a high GPA, that would also be a good sign. If she got into several schools then perhaps she is a stronger applicant than the LSAT score alone would indicate. I have never said that a person has no chance of passing the bar if he or she has an LSAT below a 145, and that would not be a true statement. Many students with scores below 145 have passed the bar. That being said, a 141 or 142 is a very, very low score. A 141 would place your friend in the bottom 15% of all LSAT takers. What I would advise your friend to do is to ask the schools that admitted her to provide her with information about how students with similar LSAT and UGPA profiles have fared at that school in the past. What has been the academic attrition rate? What has been the bar passage rate? The LSAC does free correlation studies for law schools so her schools may have one of these studies. Even if they don't, they are likely to have some internal data that would address her/your concerns. Whether the schools are willing to share that data with a prospective or admitted student may be another matter. The LSAC is very reluctant to identify a particular score below which it would be inadvisable for a student to attend law school or a law school to admit a student. In fact, the ABA standards for accreditation contain a cautionary statement from the LSAC about relying too heavily on LSATs and they advise against setting a specific cutoff score below which no student will be admitted. Another concern that I would have for your friend is that schools that tend to admit students with very low LSAT scores tend (although there are exceptions) not to have the best job placement statistics. So it is a gamble not only whether the student will graduate and pass the bar, but whether she will be able to find suitable employment upon graduation.

[anon2]

Thanks David -- much appreciated

[BoredJD]

David- Only 33% of all LSAT scorers repeat the test. You're wrong that "most" lower-tier law schools take the highest score- every law school I know of with the exception of HYS takes the highest score because that's all they have to report to USNWR. Given that the downside to retaking is a few hundred in administration fees and the upside to the "average" score increase is 2-3 points and that can be worth a lot of scholarship money, I'd say most scorers at any level are not behaving optimally, i.e., maxing retakes and study time. FWIW, my anecdotal evidence is that low scorers assume that they are just "bad standardized test takers" and thus do not put in the effort that would raise their scores. Better UG counseling is needed for those students to understand the value of studying, retaking, and putting off law schools for a few years. Furthermore, low scorers are more likely to be from more limited means and do not have the money to take expensive test prep courses. LSAC should be required to release, for free, a substantial amount of prior testing materials. (Again, the legal academy is silent on a system that discriminates against poor applicants. What a shocker).

http://www.lsac.org/docs/default-source/data-%28lsac-resources%29-docs/repeaterdata.pdf

It's possible that studying for the LSAT will make a student faster at some of the basic logical fallacies or analytical reasoning skills that are important in the profession. There's a reason physics/math and philosophy majors have by far the highest LSAT scores. Could be selection bias, could be the result of UG coursework, could be both.

Michael - You agree that there should be some cutoff for who can become a lawyer, no? Or do you really think we should get rid of the bar exam because it stops people from having the opportunity to become lawyers?

According to Western's last 509 report 33% of matriculants received no money. 42% of the students in your classes at Villanova received no money. That's about a 280K investment for those students, all in. And you do understand that most of these students aren't paying 60K/year out of their own pocket. They're paying with federal student loans, and the ABA absolutely has the obligation to make sure those funds are not being disbursed to cohorts that have a low shot of passing the bar or being able to repay them.

Having more kids go to terrible law schools, take out massive debt, and then not get jobs as lawyers isn't going to reduce elitism in the profession- it's going to exacerbate it and leave a lot of people from modest means carrying huge debt in the process.