Law School Transparency (LST) Responds to Law School Admission Council (LSAC) Press Release

Law School Transparency’s recent report, the 2015 State of Legal Education, led to numerous articles in the national media about the trend of law schools admitting large numbers of poorly qualified applicants.  This prompted the Law School Admission Council (LSAC) -- the organization responsible for the LSAT -- to issue a press release  denouncing LST’s report and purporting to rebut certain factual assertions allegedly made by LST.  The press release mischaracterizes the conclusions of the LST report. 

The press release begins:  “A report recently released by Law School Transparency (LST) has gained headlines by claiming that some ABA-approved law schools have been intentionally admitting “high risk” students who, based on their LSAT scores, do not have a reasonable chance of passing the bar.”

LST stands by the assertion that dozens of ABA-approved law schools know that they have admitted large numbers of students who, based on their low LSAT scores, coupled with commensurately low undergraduate GPAs, are at high risk of academic failure or failing the bar.  Although LST’s risk band categories are delineated by LSAT scores, LST was very careful to explain that the risks of a low LSAT score could be offset by strong academic performance in college. LST analyzed the available data and determined that law schools were not offsetting lower LSAT scores with higher GPA requirements.  Although many of these same law schools have been allocating more resources to internal and external academic success and bar prep programs, these efforts have not yet offset the overall decrease in student capability, resulting in a strong trend of decreasing bar passage rates at the high-risk category schools.

Law schools can analyze internal data about attrition and bar passage rates by LSAT score and UGPA to make reasonably accurate predictions of the likely success of applicants with similar credentials.  Law School Transparency has urged (and continues to urge) law schools to make internal data publicly accessible.  LST acknowledges that some law schools may be better at educating students with marginal predictors and helping to prepare them for the bar. LST stands by its assertion that law schools have a duty to the legal profession to share empirically validated findings.

LSAC specifically states that LST has made three false claims.  The first alleged false claim is that “LSAT scores can be used to assign bar passage risk.”  LSAC objects to LST’s labeling of students with certain LSAT scores as high risk, very high risk, and extremely high risk.  Drawing from LST’s report, LSAC notes that students labeled “high risk” at one school had a first-time pass rate of 57 percent, while comparable students from another school had a pass rate of 23 percent. Based on this disparity, LSAC concludes: “Clearly, many factors significantly affect bar passage rates above and beyond LSAT scores.”  LST agrees wholeheartedly with this statement and has never claimed otherwise.  The LSAC statement concludes “The assertion that LSAT scores alone measure comparability is patently wrong.”  Here Mr. Bernstine is arguing against an assertion that the LST report never made.  Here is what the LST report actually says about assessing risk by LSAT score: 

The [LSAT risk bank] framework represents only a starting point for assessing the risk of bar failure.  A student with a low LSAT score but very high undergraduate GPA, for example, has less risk of failing the bar than a student with the same LSAT score and a very low UGPA. Some law schools have also been more successful than others in helping students with low LSAT scores succeed on the bar exam. Where the student takes the bar exam matters as well.

While LSAC may object to “labeling” law schools and law students, LSAC doesn’t dispute the underlying premise that there is a strong correlation between LSAT score and success in law school and on the bar exam. Mr. Bernstine does not because he cannot. LST believes that “high risk” is a fair characterization of both a 43% and 77% risk of failing the bar the first time, as reflected in the data from these two schools.

The second “False Claim” identified by LSAC is:  “LSAT scores can be used to delineate risk categories.”

This is really not a different claim, but a variation on the first claim. Nevertheless, LST did use LSAT scores to delineate risk categories (minimal risk, low risk, modest risk, high risk, very high risk, extreme risk), so clearly it can be done. Mr. Bernstine’s concern seems to be that the risk categories are either arbitrary or misleading because we are making “fine distinctions.” Mr. Bernstine notes that a difference of one point on the LSAT, such as from 150 to 149, may be statistically insignificant.   But each of LST’s risk bands are more than one point wide, and represent a range of performance.  For example, the high risk category from 147-149 represents the 33^rd to the 40^th percentile on the LSAT.  The point of the risk categories is simply to illustrate that the risk of failure goes up as the LSAT score goes down, an undisputable fact that is not challenged by Mr. Bernstine.  While reasonable people could differ on exactly where to draw the various lines, the available data confirm the validity of the risk categories as presented in the report. Again, LST invites any law schools with data tending to refute the validity of these categories to come forward and make that information publicly available.  We want to be wrong. 

In recent weeks, several deans at high-risk schools have made comments in the media suggesting that there is no correlation between LSAT scores and bar passage at their law school, but none have provided data to support their assertions.  The most clearly false claim came from Dean Penelope Bryan at Whittier Law School. She told the Los Angeles Times that “[t]he LSAT score has no predictive value for the success of Whittier Law School students on the bar exam.”  Incidentally, Whittier’s first time bar pass rate in California dropped from 64.7% in July 2013, to 42.7% in July 2014 to 30% in February 2015. (Note - Whittier had only 10 first time test-takers for that administration; California's July 2015 results by school are not yet available, but the statewide results were lower.)

At the same time, Whittier’s entering class profile progressively weakened.  In 2014, over half of the entering class at Whittier was at 146 or below on the LSAT.  To Whittier’s credit, the school dramatically shrank their first year entering class in 2015, substantially raising the LSAT profile of the bottom half of the class, with the 50th percentile increasing from 146 to 148 and the 25th percentile increasing from 143 to 146.  Clearly, Whittier has recognized that LSAT scores do matter, even if Dean Bryan won’t publicly admit it.

Mr. Bernstine’s third objection to the LST report is our alleged claim that “A study based on 25-year-old data can be used to assess current bar passage risk.”  Mr. Bernstine makes much of the fact that LST cited a landmark LSAC study, the LSAC National Longitudinal Bar Passage Study, in support of the uncontroversial assertion that there is a strong correlation between LSAT scores and bar passage. He notes that the LSAT has changed since that study was published, but, importantly, he does not assert that the current version of the LSAT has any less of a correlation than the old version used in the study. Neither, to our knowledge, has the scaling and equating process changed in a way that damages the link between current tests and previous tests for comparability. And, of course, the LSAC study was just one of several data sources in support of the report’s conclusions.

The LSAC release concludes by noting the limitations of the LSAT and scolding LST for allegedly failing to identify these limitations in our report.  Ironically, the LST report specifically, correctly, and adequately addresses each of these issues. Despite these caveats, the fact is that LSAC has developed a highly accurate test for assessing law school aptitude.  Because the bar exam is essentially a cumulative law school exam, it should not be surprising that the LSAT turns out to have a strong correlation with bar passage as well.  The irony is that while LSAC spends millions refining the LSAT each year to ensure that it is the best predictive tool available for law school success, LSAC feels compelled to downplay the validity of the exam.

LST understands that LSAC is speaking on behalf of many of its customers, which include the scores of law schools identified by LST as high risk, very high risk and extreme risk schools.  LST understands that many of these law schools are very unhappy being called out for their reprehensible admissions practices.  Undoubtedly, being labeled an “extreme risk” school is not good for business, and does not make the school’s alumni very happy. 

LST should not be criticized for pointing out uncomfortable truths.  Dozens of ABA-accredited law schools are making reckless choices that have a huge impact on thousands of real people. If effective labeling motivates this minority of law schools to act responsibly, then the hurt feelings of a few people associated with these schools is a small price to pay.   Until LSAC, or the law schools identified by LST as having problematic admissions practices, come forward with actual data to refute the conclusions in the LST report, general attacks on the validity of the report should be taken with a large grain of salt.

 

Kyle McEntee, Executive Director, LST                                                  David Frakt, Chair, LST National Advisory Council

Comments

[Sy Ablelman]

Get rid of all these over paid law deans and academics. There are thousands of underemployed veteran Solos with REAL court experience out there earning an average of 37K (See Paul Campos) like me who would kill for an academic gig. I am willing to accept a DOUBLE class load at any law school and will accept a salary of 50K plus Health care. That will lower tuition and debt burden. Plus I will teach these kids practical law. How to get a client out of jail. How to get a mope back his driver's license. How to strong arm ("extort") a fee out of gang banger street dope peddler. And how to coerce a cash bond refund from a tatted up A-hole charged with Domestic Battery for beating up his baby momma. That client doesn't give two shits about Penoyer and International Shoe, because "She hit me first" and "She's going to drop the charges." We Solos will teach you how to get that cash bond money into YOUR POCKET, where it belongs.

[Milan]

Every piece of research I have seen suggests that the LSAT is weakly correlated with bar passage. See, for example, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2308341&download=yes. Kyle and David, what is the basis for your claim that "there is a strong correlation between LSAT score and success . . . on the bar exam?"

[Sy Ablelman]

One more thing, you law deans are turning our investments (JD's) into turds by accepting everybody and anybody in the name of serving the UNDERSERVED. Neighborhood Solos like me and small firms have been doing that for decades, and I will even appear in court for $100. Open any Popular Mechanics and Popular Science magazine from the 50s and 60s and you will see ads from Blackstone College of Law and La Salle Extension University. Those "schools" promise to make one a "Law Trained Man" by reading the law a few hours a week. They are right next to the bulldozer operator and carpet cleaning franchise ads. There is an element of hucksterism to them. I find them inescapably similar to many law school websites with a bunch of "APPLY NOW" buttons and promises and hefty and promotions.

[Anon]

Sy Ableman, you are a savvy guy, but you are missing a simple point, which is the reason you will never be hired to a tenured law professorship: the purpose of law schools is not to teach future lawyers how to make a living. The purpose is to enrich the people who work at the law schools.

[Sy Ablelman]

I am not only savvy, but good looking too!

[Deborah Merritt]

I have read all of LST's report, which is very detailed and thoughtful. It makes exactly the distinctions that LSAC claims were omitted. I'm shocked at the disingenuousness of that LSAC press release; it has no place in serious discussion of these issues.

I don't personally know any legal educator who doubts the correlation between LSAT scores and bar passage (both first-time and eventual). Indeed, the responsible educators I know devote considerable thought to which applicants might be able to succeed despite low test scores--and how to offer them the education that will help them achieve that result. If all of the schools admitting large numbers of low-scoring applicants are doing that then, as Kyle says, more power to them. *But* bar passage rates suggest otherwise.

If anyone has doubts about the correlation between LSAT scores and bar passage, look at Mark Albanese's article in the June 25 issue of the Bar Examiner. (I would offer a link but I'm afraid that will send this post to the spam pile.) It describes a very strong correlation between LSAT scores and MBE scores--which in turn drive bar scores and pass rates.

Professionals are supposed to regulate themselves and other members of their group; that's what distinguishes us from other occupations. The question here is whether law schools are willing to look seriously at their practices (both individually and collectively) with respect to applicants. Do we genuinely believe that all of the applicants schools are admitting have a reasonable chance of passing the bar and serving clients effectively? Even if it's worthwhile to take risks on some applicants, how many risky students can each school reasonably support?

[David Frakt]

Here is a link to the Article referred to by Debbie Merritt. It is fascinating reading and provides very strong support for the conclusions reached in the LST report.

http://www.ncbex.org/pdfviewer/?file=%2Fassets%2Fmedia_files%2FBar-Examiner%2Farticles%2F2015%2F840215-TestingColumn.pdf

Happy Reading!

[David Frakt]

Milan - There is a significant body of research proving that LSAT Scores are strongly correlated with bar passage rates. In addition to the article cited in the above, there was a a very detailed report prepared by the NCBE for the The New York Board of Law Examiners in 2006 with a large sample size . The authors found “The correlation of bar examination scores with LSAT scores is fairly high”
http://www.nybarexam.org/press/ncberep.pdf (See Section 5)

A 2004 article in the California Western Law Review states: “There is an extremely high correlation between LSAT score and bar exam result.”
http://scholarlycommons.law.cwsl.edu/cgi/viewcontent.cgi?article=1143&context=cwlr

Gary Rosin’s 2008 article http://papers.ssrn.com/sol3/papers.cfm?abstract_id=914224
“LSAT scores of a law school’s entering classes were the most significant factor in determining its Bar passage rate.”

Stephen P. Klein & Roger Bolus, Analysis of July 2004 Texas Bar Examination Results by Gender and
Racial/Ethnic Group 12 (Dec. 15, 2004) (LSAT scores strongly correlates with bar passage)

The article that you cite had a very small sample size of two sets of bar exam takers at one law school. It found a correlation between LSAT score and bar passage but asserted it was not as strong as other factors. The research methodology and sample sizes in these other studies provide compelling proof of the strong correlation. Although an LSAT score is not destiny, it is a very useful data point in predicting the risk of failure on the bar.

[Anon]

Are you or are you not the same D. Frakt who failed to secure the deanship of the very same law school you now attack?

And why would you deny law school to black and hispanic applicants who have lower GPAs but can achieve acceptable bar passage rates if given a chance, as M. Simkovic explained today with the following comment:

There may be legitimate concerns about long term eventual bar passage rates for some law students. However, "Law School Transparency's" back-of-the-envelope effort, focused on short term outcomes, does not provide much insight into long-term questions. The most rigorous study of this issue to date--the LSAC National Longitudinal Bar Passage Study--concluded that "A demographic profile that could distinguish first-time passing examinees from eventual-passing or never-passing examinees did not emerge from these data. . . . Although students of color entered law school with academic credentials, as measured by UGPA and LSAT scores, that were significantly lower than those of white students, their eventual bar passage rates justified admission practices that look beyond those measures."

It is particularly distressing that D. Merritt who teaches at a public flagship law school would endorse an "analysis" which is little more than elitist ideology aimed at closing off the legal profession to underrepresented groups in our society.

[David Frakt]

Dear Anon,

As readers of this blog well know, I was one of 7 candidates invited to Florida Coastal School of Law as a finalist for the Deanship there in the Spring of 2014. As part of my visit, I was invited to give a presentation to the faculty and staff on my vision for the future of the school. In my presentation, I raised precisely the same concerns raised by Law School Transparency in the 2015 State of Legal Education report. I noted that the school had dramatically lowered admission standards and predicted that this would lead to plummeting bar passage rates. In fact, I specifically predicted that the first time passage rate for the July 2014 bar would drop below 60%, a prediction that proved to be accurate. I suggested that the school was placing profits over the interests of students by admitting large numbers of unqualified applicants. I indicated that if I was selected to be the Dean that I would immediately reverse the current admissions policies, and dramatically cut the class size in an effort to raise the bar passage rate and place the school on a sustainable long-term footing. These ideas were well-received by much of the faculty, but outraged the school president, Dennis Stone, who barged into the lecture hall and threw me off the school property. It was this experience that reinforced my resolve to do something about the predatory admission practices of schools like Florida Coastal through writings on the Faculty Lounge and through service to Law School Transparency. Although I suppose one could say that I "failed to secure the deanship" at Florida Coastal, it might be more accurate to say that Florida Coastal failed to secure a dean, as the spring 2014 Dean search failed. None of the seven invited candidates took the job. (I don't know if it was actually offered to any of them.) The school eventually offered the Deanship to a faculty member after readvertising the position in the fall of 2014.

In response to the second part of your question, I am not aware that I have attacked any particular law school, at least not recently. (Perhaps you are confusing me with the New York Times Editorial Board, which recently described Florida Coastal's admission practices as "a scam." http://www.nytimes.com/2015/10/25/opinion/sunday/the-law-school-debt-crisis.html

Incidentally, I did not write the LST Report, although I did contribute to it and endorse its findings and recommendations. The report identifies 74 ABA accredited law schools that admitted classes consisting of at least 25% at-risk students. Of those 74 schools there were 26 which were categorized as extreme risk, 19 as very high risk, and 29 as high risk. All three InfiLaw schools were in the extreme risk category based on the profiles of students admitted in 2014.

Neither I nor LST has suggested that any law school deny admission to qualified minority students. Our analysis is entirely race-neutral. We have simply urged that schools be required to comply with the ABA standard which requires that law schools only admit students with a reasonable chance of success in law school and on the bar exam. In our view, admitting hundreds of students with a combination of extremely low LSAT scores and very low undergraduate GPAs is inconsistent with this standard. At the very least, such an admission policy merits close scrutiny from regulators to ensure that there is empirical data supporting these admission decisions. I, and LST, have repeatedly called on law schools to provide the internal data which would justify admitting so many high risk students. No law school has yet taken us up on this invitation.

[Anon]

Well now that you've come clean here I assume you will place appropriate disclosure on the LST website as well? After all you have nothing to hide...although of course we have only heard one side of your FCL story - and honestly why would you think you had a chance to get a job running a for profit law school with a plan to cut their revenue?

As for your suggestion of race neutrality - that sounds strangely familiar...hmm oh wait a minute I remember - it was the world of public elementary and high schools prior to 1954! Pleeeze - peddle this nonsense somewhere else.

[Jojo]

Anon,

Ad hominem attacks on Frakt and Merritt (and LST, and Kyle, and ...) plus race baiting and invocations of Plessy do not make your argument more persuasive. I do not know Michael Simkovic, but I have my doubts that he'd be too excited to count you as his booster.

As for the instant battle in this law school reform proxy war, let's accept as true that there are diamonds in the rough who will thrive as lawyers despite sub-150 LSAT scores and sub-3.0 undergrad gpas. How many are we talking about? To my knowledge no one is advocating a hard-bar minimum scores for enrolling. I can imagine non-native English speakers, or bright grinders with dyslexia, or some individuals with a tough lot growing up, who though struggling in college, will manage to be competent or fine lawyers or public leaders. But we're talking small numbers here, not half the class.

And, just to further call out the nonsense, that's always been the case! No one was pushing to abandon academic selectivity prior to 2012. Why not? Everyone knows what's going on here. The music stopped and there aren't enough chairs. Schools that once tried to teach jurisprudence and the Langdellian case method to future judges now are trying to justify admitting students with a median lsat score that approaches a full standard deviation below the mean of all lsat takers (not just matriculants). Almost anyone who wants to go to law school, now can. Is that a good thing? Does it come with consequences to the students (debt, lack of jobs, unrealistic expectations of what lawyers actually do), the bar (saturation, inability to make a living, inability to obtain mentoring/experience/competence), the schools (reputational, alienated alums), the public (lower quality of representation/ethical problems from indebted or desperate practitioners).

America needs and will need good lawyers. It needs and will need a finite number of them, however.

Students who might make bad lawyers or law students might make great actuaries, shop owners, mechanics, marketers, roofers, metal workers, herbal medicine developers, etc.). It's not like the choices are law school or the gallows. Be honest about what's been happening in law. If the median graduate from the median school can't make a long-term run of practicing law, something is amiss with our system. If that same median/median can't 5 years out earn at least $65,000 plus health insurance, there's a real problem. A middling law student was until very recently an above-average Joe/Jane with a college degree. If that person can't earn 65k five years out, then the law school juice isn't worth the squeeze from an economic perspective given the cost to obtain a degree and the alternative employment outcomes available.

[[M][@][c][K]]

Gosh Anon at 12:55 (and numerous other intemperate posts.)

You sarcastically attack David Frakt for not getting hired as dean at several law schools. Have you considered that this fact speaks volumes about the selection criteria for those law schools (are they looking for lickspittles, "get along Harrys," and other resolute defenders of the status quo.)

And since you are into speculation ad hominem attacks on Frakt, Merritt, McEntee, etc. isn't it fair to speculate about your personal circumstances....speculation that is rendered very easy by the hysterical nature of your posts.

[anon]

How is LST funded? It is just hard to believe that it is just an expensive hobby for a few graduates from Vanderbilt. Frakt put out a call to join his "national advisory council". It is hard to take such a request seriously unless LST is transparent, itself.

[Milan]

David,

Thank you. Just to clarify, is your claim that for a given candidate, LSAT is a strong predictor of MBE score? I think that one has to be careful to distinguish studies that analyze group results and individual results.

[David Frakt]

Dear Anon,

If you would care to identity yourself and provide your true e-mail, I would be happy to respond to any legitimate questions that you may have.

I am pleased to report that my call for applicants for the National Advisory Council, along with a number of targeted invitations, received a tremendous response. LST will be announcing the membership of the National Advisory Council in the very near future. The NAC includes law professors, current and prior law school administrators, pre-law advisors, consumer advocates and several practicing attorneys and is diverse in several ways.

[David Frakt]

Sure. After 9

Sent from my iPhone

[anon]

I asked a legitimate question, and a simple one. How is LST funded? I'm not sure why revealing my identity would change the answer or need to take it off-line.

[Leo]

Wow. So now those supporting the status quo are reduced to calling those who disagree with them racists!! This must be the beginning of the end for some schools.

[terry malloy]

I donated 200$ to LST when they were raising money on Fundrazer for server fees and travel expenses for the executive director. they collected about 3 grand.

I'm pretty sure it isn't the Koch brothers.