Search the Lounge

Categories

« Kweku Adoboli Speaks! | Main | Kweku Adoboli’s Future Career In Music Promotion? »

October 25, 2015

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Matt

You claim to have confirmed the reliability of your model by checking how frequently people pass the bar on the first try. That seems irrelevant. Many great lawyers, including supreme court justices and governors, failed the bar on the first try.

Jess

I do no think being a governor is related to being a lawyer, indeed a lot of governors are not lawyers. I had not heard Supreme Court justices failed on the first try -- did you mean US Supreme Court, and if so, which ones?

David Frakt

First-time bar passage rate is widely considered to be a useful benchmark, and that is what the ABA requires that law schools report. So, for most schools, that is the only data that is available. It may also be useful for prospective law students to consider an "ultimate" or "eventual" bar passage rate, but considering the devastating consequences of failing the bar, financially and emotionally, I believe the focus is appropriately on first-time pass rates. Also, in my opinion, the more times a graduate takes the bar before passing, the less that passing the bar should be credited to the law school. The LST Report does include "eventual passage" rate for the anonymous school, but not for Denver. For those in the extreme and very high risk groups at the anonymous school, the eventual pass rate was 36% and 38%, respectively. Not very good. And schools are admitting students who are far weaker than the students who achieved these abysmal results. No law school has a 100% pass rate. Some people who are undeniably capable of passing the bar fail it, either due to a case of nerves, or being ill on the day of the exam, or, more commonly, not studying hard enough. The issue is whether schools should admit students that will likely never pass the bar no matter how hard they study.

Mark Edwards

Here's a question: what should the bar passage rate be? It's tempting to say 100%, of course, but they can't be right -- that would mean graduates were being challenged enough. The passage rate for the CPA exams is generally much lower than the average bar exam passage rate (https://media.nasba.org/files/2015/01/2014_Overall.pdf). An average performing university CPA program would probably be listed as an extreme risk under the metric used here.

On the other hand, medical board exam passage rates look an awful lot like bar exam passage rates, including the precipitous decline in the last five years. (http://knowledgeplus.nejm.org/abim-pass-rates-behind-declines/). Obviously, that can't be explained by lower entering LSAT scores, so I wonder if there is a common cause we haven't discovered yet.

Food for thought, anyway.

Mark Edwards

Here's a question: what should the bar passage rate be? It's tempting to say 100%, of course, but they can't be right -- that would mean graduates were being challenged enough. The passage rate for the CPA exams is generally much lower than the average bar exam passage rate. An average performing university CPA program would probably be listed as an extreme risk under the metric used here. On the other hand, medical board exam passage rates look an awful lot like bar exam passage rates, including the precipitous decline in the last five years. Obviously, that can't be explained by lower entering LSAT scores, so I wonder if there is a common cause we haven't discovered yet.

David Frakt

Mark -

Thanks for your questions. I don't know anything about medical exam boards. I know a little about CPA exams because my wife is a CPA. I don't think the CPA exam is a good comparison to the bar exam because people can't sit for the CPA exam until they have significant experience in accounting, and CPA candidates generally take the exam while they are already working full time. The CPA exam is broken into parts which are taken at different times. And while failing a section is common, it is also not a big deal. You don't lose your job. You just sign up and take it again a few weeks later. It costs a few hundred bucks, but is not a life changing event like failing the bar exam often is.

As for the question of what the bar passage rate should be, the LST report recommends changing the ABA standard so that a law school must have an 85% percent pass rate within two years (four administrations of the bar exam) after a student graduates. (According to the LSAC National Longitudinal Bar Passage Study, 99.9% of people who pass the bar exam have done so by the fourth attempt. After three attempts, the figure is 99.3%.) I believe this is a reasonable standard. In addition, I believe that ABA-accredited schools should be required to be within 15% of the state average, or have a 60% first-time bar pass rate, whichever is higher. Schools that drop below this standard for a calendar year (combining the February and July bar results) should automatically be placed on probation, absent extraordinary circumstances (like a large number of students from the top of the class transferring to other schools where they passed the bar on the first attempt). I do not consider it acceptable for an ABA-accredited law school to have a bar pass rate below 60%, even in jurisdictions with tougher standards, but there are currently several schools with rates in the 50s and even 40s that are still meeting the current ABA standard. Many of the students that failed the bar at these schools never should have been admitted in the first place because there was no evidence that they had reasonable aptitude for the study of law.

Jojo

I'm among the harshest critics of law school, but I think 85 percent is too high. I'd do 75 though. If 3 out of 4 cannot pass the bar in 2 years, your school is doing something wrong. Either you aren't teaching them the nuts and bolts of law or else you are admitting too many students who you know to be at risk (i.e. Scamming). 75 percent gives schools enough cushion to take a risk and admit a few possible diamonds in the rough each year without risking accreditation.

I'd also make my cutoffs have teeth. After 1 year you are on probation. If you get 2 more strikes in the subsequent 3 years, you lose accreditation.

Anonymous

Another option is reform the attorney licensure system. The main skill set necessary for passing the bar exam is the ability to cram for exams. As many have noted, lawyering never involves free writing for six hours at time without the opportunity to look up statues or laws. Similarly, practice never presents attorneys with multiple choice exams. Yet these are the two activities from which the bar exam derives its scores and determines the fate of thousands. Does that really make sense?

If anyone can cite any studies that show a correlation between effective lawyering and bar exam scores, that would be something. Instead, we rely on a century old process that is just as likely to reward an idiot savant as it is to punish someone without test-taking skills but who has the empathy and ability to be a damned good lawyer.

Note: this isn't sour grapes, I passed two bar exams. I just think the bar exam is an illogical barrier to practice.

anonymous

No study seems needed. Being a competent lawyer requires being a lawyer.

RQA

The LST report focuses, as does this discussion, on private and especially for-profit schools. But two categories of public schools stand out as problematic: historically black universities (the schools with the riskiest admissions) and universities in remote rural areas (viz., the Dakotas). I'm not sure how to think about the issues those schools present for this analysis: do they affect at-risk populations particularly in need of protection, or should we assume the schools' admissions policies have more benign motivations and should be given the benefit of the doubt? Either way, I'd want a policy placing schools on probation to consider which schools would seemingly be most likely to be involved or explain why and how they'd be treated differently.

Paul Campos

"The main skill set necessary for passing the bar exam is the ability to cram for exams. As many have noted, lawyering never involves free writing for six hours at time without the opportunity to look up statues or laws. Similarly, practice never presents attorneys with multiple choice exams. Yet these are the two activities from which the bar exam derives its scores and determines the fate of thousands. Does that really make sense?

If anyone can cite any studies that show a correlation between effective lawyering and bar exam scores, that would be something. Instead, we rely on a century old process that is just as likely to reward an idiot savant as it is to punish someone without test-taking skills but who has the empathy and ability to be a damned good lawyer."

Hmmm, I wonder if there's any other part of the required process for becoming a lawyer to which this very same criticism applies?

David Frakt

I would not endorse creating a separate standard for historically black universities. The admirable goal of promoting greater diversity in the profession is not an excuse for admitting students with exceedingly dim prospects of becoming lawyers. While it may be slightly less morally reprehensible to admit unqualified students for the purpose of keeping a worthy institution with an important mission open, as opposed to admitting unqualified students for the purpose of lining the pockets of the owners, it is still fundamentally wrong to exploit such students.

Barry

Matt: "You claim to have confirmed the reliability of your model by checking how frequently people pass the bar on the first try. That seems irrelevant. Many great lawyers, including supreme court justices and governors, failed the bar on the first try."

Many people have survived horrific battles.
Many people have survived falling from great heights.
Many people have survived being bitten by poisonous snakes.
Many people have [insert her]

Mark Edwards

It seems like many people agree that the bar exam is not a very useful measure of ability to practice, for fairly obvious reasons -- the exam format in no way resembles the practice of law.

But if that's true, then can the answer to this problem really be to only admit students with high LSAT scores -- high scores on an exam whose format in no way resembles the practice of law?

David Frakt may be correct that there is a high correlation between LSAT score and likelihood of bar exam passage. But that suggests that both the LSAT and the bar exam are measuring the same thing -- the ability to perform well on an exam with a format that in no way resembles the actual practice of law.

Many law schools and profs are implementing new forms of assessment that at least bear a passing resemblance to practice. That seems very wise to me. So that shouldn't change.

It seems like we've stumbled into a 4 step process for becoming a good lawyer that makes no logical sense:

1. Do well on a standardized exam that in no way resembles the practice of law.
2. Either (a) attend a classically Langdellian law school, and do well on exams that in no way resemble the actual practice of law; or (b) attend a school with a more practically-oriented means of assessment, and do well on those assessments.
3. Do well on a standardized exam exam that in no way resembles the practice of law.
4. Practice law well.

The best solution to this problem is clearly NOT to get rid of institutions described in 2(b) above. But, ironically, because such institutions tend to be lower prestige, they are the very ones that are composed of students with low LSAT scores. And it is true, as David Frakt points out, that low LSAT score is correlated with low bar passage rate.

If we do as many people suggest, and close down the institutions with low LSAT scores and low bar passage, then we compound the massive gap between law school and practice of law. If, on the other hand, we de-emphasize LSAT score and seriously reform assessment for purposes of bar admittance, we close the gap between law and practice. In either case, we are likely to have fewer people attending law school who discover that after all their investment of time and money they are unable to practice law.

But I would very much prefer we alter steps 1 and 3, rather than shut down institutions described in 2(b). I have a strong sense that the practicing bar would prefer that as well.

Mark Edwards

I'm hoping my previous comment shows up, because this is an addendum to it . . . .

In the meantime, it would be very interesting to compare actual bar passage rates to the predicted rates using David Frakt's metric for all law schools. We could then compile a ranking of schools based on the extent to which their graduates exceed bar passage rate predictions (or not) based on their LSAT scores. If we are still going to use the LSAT and the bar exam, it would be useful to know which schools are the most transformative for their students. Under the current system, arguably, those schools are doing the most effective teaching.

For example, it's possible that there are 2 schools, one with an expected bar passage rate of 40% that is producing 80% passage, and another with an expected bar passage rate of 95% that is producing 85% passage. Under the LST proposal as I understand it, the first would lose accreditation and the second would not. That seems like a very bad result to me, as we would be losing effective teaching and keeping ineffective teaching.

RQA

Perhaps the ultimate answer should be the same for law schools at historically black universities as for for-profit law schools, but I think the analysis has to be significantly different. Were every school on the LST list of 18 with the 50th percentile at "extreme" or "very high" risk to close, we'd lose only three for-profit schools, but every (all four) historically black school outside D.C. I'm inclined to believe that prospective students who then wouldn't be able to attend Arizona Summit, Florida Coastal or Charlotte would either have other decent choices or shouldn't have been going to law school in the first place. I'm not so sure about the prospective students who wouldn't be able to enroll at North Carolina Central, Florida A&M, Southern University or Texas Southern. I recognize that some of the students who'd be displaced should not be in law school at all. But I'm not sure the students who might stand to benefit from law school would be equally well-served by the majority-white options they'd be left with. (I know the three for-profits aren't exactly majority-white either, but they're a far cry from the historically black schools.) I'm harping on this because it seems to me LST is ignoring the impact of its recommendations on historically black schools. And, to reiterate, I'm not sure those schools shouldn't be drastically reformed or closed -- but I think the costs associated with that outcome would need to be weighed very carefully and quite separately from the effect on for-profit schools.

Anon

I appreciate efforts to collect and analyze data relating to law school. More information is a very good thing so kudos on that. But on your "reliability" test, I do not understand why you would draw such generalizations on reliability from a sample of TWO schools. It's just not representative and your attempt to discuss potential variables is very weak and incomplete. Why can't you just note the data and let the reader decide what to make of it? I can think of other school couples that you could have used where the results would have differed significantly. I think you weaken your argument by going a few steps too far.

David Frakt

Anon -
I am not drawing generalizations based only on two schools. Keep in mind that I developed my model based on my review of several years of data at Western State University College of Law, and by reviewing the available research. I also taught at Barry Law School for two years where I observed similar results. My father was formerly Dean of Loyola Los Angeles and Widener University Law School and he observed similar results. The data from these two schools confirms what I already knew, but which some of my critics, such as Jay Conison at Charlotte Law School, have vehemently denied. What is significant is that this is the first hard data that we have from any law schools that has been made publicly available. If other law schools are able to produce better outcomes with students with similarly low entrance credentials, we would expect those schools to want to publicize that information, and it would be very useful for prospective students to have that information. The main thing that I, and LST, are calling for, is greater transparency and enforcement of existing standards. Schools should provide empirical data to the accrediting agency (the ABA) justifying their admissions policies. If they are admitting students that they know to have a 10 or 20% chance of completing law school and passing the bar, then they should be forced to publicly acknowledge and defend that practice (if they can). Whatever your view of the efficacy or value of the bar exam, it seems highly unlikely that it is going to go away or be radically reformed anytime soon. The students being admitted to law school are almost certainly going to have to take a bar exam very similar to the one that droves of students just failed in July. And, unless states dramatically lower the score required to pass, then record percentages of graduates in 2016, 2017, and 2018 are going to fail. I view this is a large-scale, but completely preventable, human tragedy. LST's proposals would not necessarily result in the closure of any law schools, but the proposals would require law schools which are currently exploiting unqualified students to stop doing so. This would likely mean getting dramatically smaller and laying off some employees until law school applications rebound. If a school can't stay in business without exploiting unqualified law students, then it is no great loss to society if that school is forced to close.

twbb

Anon, in statistical terms it's not a sample of two schools, because the inference is not being drawn about the schools but about the students.

anon

The fact that so many who run and teach at law schools have never been or wanted to be lawyers is proof enough that the system is a crooked one. This needs to change. The ones losing here are students who seek nothing more than a move up the socioeconomic ladder and have a fulfilling career.

The comments to this entry are closed.

StatCounter

  • StatCounter
Blog powered by Typepad