Conison Guest Post

The following is a guest post from Jay Conison, who is Dean of Charlotte Law School. It responds to David Frakt's recent post on lsat scores as predictors of success:

The Charlotte School of Law, like many other schools, loses students through transfer. Transfers from Charlotte are not as severe a problem as at many other schools, but they are still disappointing. In recent years, we have increasingly been losing students at lower LSAT ranges. In 2013-14 students from Charlotte transferred to schools that included University of Texas, Notre Dame, Case Western, Connecticut, Chapman, American, and Cincinnati. Many of these students had LSAT scores ranging below 145; few had LGPAs near the top of their class.  (I put the information this way to shield individual student information.) The facts are similar at Charlotte’s sister schools, Arizona Summit and Florida Coastal.

I make this point because Mr. David Frakt purports to be an expert on legal education and in particular the relationship between LSAT and educational outcomes. He has asserted that no one with an LSAT below 145 has more than a trifling chance of passing a bar examination, and that schools that admit students with such low LSATs are deceiving students and the public. Mr. Frakt has chosen in particular to attack the Charlotte School of Law, the Florida Coastal School of Law, and the Arizona Summit Slaw School. The crux of his attack is the relatively low LSAT profile of the students. But if Mr. Frakt is correct in his claim that law schools should not admit any student with an LSAT below 145, then the transfer data would suggest quite a massive conspiracy against law students, a conspiracy that includes many schools considered pre-eminent, if not elite. I am not aware of any such conspiracy. However, I can shed light on how Mr. Frakt has gone astray. 

LSAT scores have two functions for law schools. One is as a measure of prestige. For many years schools could be pretty confident that most admitted students would succeed, and so could be focused less on the predictive value of the LSAT and more on its prestige effects. Schools aggressively competed for the highest LSAT profile as part of competition for high prestige. It is true that there is no logical necessity for LSAT being a measure of law school prestige, any more than for shiny beads measuring societal status, but that is the way the world is. For a very long time, the prestige function of LSAT was the chief concern of schools in utilizing the LSAT, even if it was rarely admitted.

 This function of LSAT, however, has been put under severe stress by the sharp downturn in applicants. Many schools want to simultaneously publish a high LSAT profile, in order to continue to compete for status, but also maintain the high enrollment needed to provide large operating revenues. This tension has led many schools to reduce first-year class size, in order to preserve LSAT profile and thus prestige, while engaging in aggressive transfer admissions to build revenues from students whose LSAT scores are not counted (in U.S. News and other measures of status). As a result, first-year LSAT profile no longer accurately represents the real student body LSAT profile.

The other function of the LSAT—and the one for which it was designed—is as a predictor of academic success. While it does have predictive value, an important question is just how good, and that varies from school to school. Charlotte uses the LSAT only as a predictor of academic success, and not as a measure of prestige. Charlotte also understands the limits of the LSAT as a predictor, and understands that it is possible to have other, even better, predictive models using other forms of assessment. One of the most important additional tools Charlotte (and several other schools) use is an admissions program by which students with lower LSAT and UGPA enroll in two challenging law school equivalent courses (criminal procedure and negotiable instruments). If their assessments in this program are high enough, are admitted to the J.D. program.

As the applicant pool has become smaller, Charlotte has relied more on the programmatic method to aid in admissions decisions. This has posed an unexpected challenge, in that many lower-LSAT students to whom we have offered entry into this alternative admission program have been accepted outright by other law schools (sometimes with scholarships). Because we use this program and most other schools do not, there is a difference between the predictive value of a lower LSAT at Charlotte and what it represents at another school that relies solely on LSAT and UGPA for predictions of student success.

Thus, it is wildly over simplistic to look—as Mr. Frakt does—only at first-year LSAT profile and believe that it tells a dispositive story about a law school. LSAT profile is not destiny, and it is not now (if it ever was) an unequivocal indicator of aggregate likelihood of academic success, because many schools hide their lower LSAT students from public reporting, and because some schools, like Charlotte, utilize more comprehensive, validated predictors of success.

There is much more I could share about our institutional ability to increase the chance of student success, such as our continual work to strengthen the predictive value of the alternate admission system; our very high investment in academic support and bar preparation (e.g., currently 11 faculty and an Assistant Dean solely dedicated to these educational services); a large writing lab integrated into the Library; and more.  Mr. Frakt’s view rests on model of a law school as just a black box, into which one inputs LSAT scores and outputs bar passage. But as anyone in a law school knows, it is no such thing. Rather, it is a complex system for delivery of educational services, the details of which are very important for assessing the quality and value of a school.

Comments

[anon]

Passing rate of first-time test takers on the 2014 North Carolina July bar exam:
1. UNC: 87% (138 of 159)
2. Campbell: 86% (119 of 139)
3. Wake Forest: 79% (75 of 95)
4. Duke: 79% (19 of 24)
5. N.C. Central: 76% (72 of 95)
6. Elon: 69% (49 of 71)
7. Charlotte: 56% (103 of 184)
NC Total: 75% (575 of 767)

Passing rate of first-time test takers on the 2014 Florida July bar exam:
1. Florida: 90.6% (241 of 266)
2. Florida State: 81.8% (162 of 198)
3. Miami: 81.2% (233 of 287)
4. Stetson: 78.2% (158 of 202)
5. Florida International: 78.2% (86 of 110)
6. FAMU: 73.6% (89 of 121)
7. Nova Southeastern: 69.7% (161 of 231)
8. St. Thomas: 69.2% (101 of 146)
9. Barry: 62.0% (85 of 137)
10. Florida Coastal: 58.0% (149 of 257)
11. Ave Maria: 56.6% (43 of 76)
FL Total (Florida law schools only): 74.2% (1508 of 2031)

Passing rate of first-time test takers on the 2014 Arizona July bar exam:
1. Arizona: 88.6% (70 of 79)
2. Arizona State: 88.4% (122 of 138)
3. Arizona Summit: 54.7% (86 of 157)
AZ Total: 74.6% (421 of 564)

[anon]

Passing rate of first-time test takers on the 2014 North Carolina July bar exam:
1. UNC: 87% (138 of 159)
2. Campbell: 86% (119 of 139)
3. Wake Forest: 79% (75 of 95)
4. Duke: 79% (19 of 24)
5. N.C. Central: 76% (72 of 95)
6. Elon: 69% (49 of 71)
7. Charlotte: 56% (103 of 184)
NC Total: 75% (575 of 767)

Passing rate of first-time test takers on the 2014 Florida July bar exam:
1. Florida: 90.6% (241 of 266)
2. Florida State: 81.8% (162 of 198)
3. Miami: 81.2% (233 of 287)
4. Stetson: 78.2% (158 of 202)
5. Florida International: 78.2% (86 of 110)
6. FAMU: 73.6% (89 of 121)
7. Nova Southeastern: 69.7% (161 of 231)
8. St. Thomas: 69.2% (101 of 146)
9. Barry: 62.0% (85 of 137)
10. Florida Coastal: 58.0% (149 of 257)
11. Ave Maria: 56.6% (43 of 76)
FL Total (Florida law schools only): 74.2% (1508 of 2031)

Passing rate of first-time test takers on the 2014 Arizona July bar exam:
1. Arizona: 88.6% (70 of 79)
2. Arizona State: 88.4% (122 of 138)
3. Arizona Summit: 54.7% (86 of 157)
AZ Total: 74.6% (421 of 564)

[terry malloy]

Jay, how does it feel to male your living monetizing the poor decisions of youths not intelligent or wary enough to see through your polished sales pitch?

Vultures have more honor.

Shame.

[Nathan A]

What a load of baloney.

Look, I don't think anyone views a person's LSAT as a singular predictor of that person's density. If Infilaw had some Super Awesome Admissions Machine capable of identifying diamonds (people capable of succeeding as lawyers) in the rough (LSAT =< 142, CSL's current LSAT median), then there would be almost no change in the NUMBER of students with low LSAT scores (LSAT =< 142) who were accepted by CSL between 2010 and 2014.

[Anon]

If the AAMPLE program is so great at determining future success, do all students have to take it? Or just low LSAT/GPA admits? Does Charlotte charge tuition? How much? What percentage of AAMPLE students are admitted now, and in 2010 or 2012?

BTW: quoting you: "first-year LSAT profile no longer accurately represents the real student body LSAT profile." I believe the same could be said for Charlotte, as higher LSAT students transfer out. That begs a question: What is the LSAT profile of the graduating class of 2014?

Finally, isnt it implausible that the profit motive of Infilaw/Sterling has no effect on the decisions of who to admit as a student to CSL, FCSL, or ASSL? And no, I simply dont believe that "non-profits are just as greedy as for-profits." Having seen the behavior of for-profits, that is simply untrue.

[John Steele]

From Dean Conison's bio: "He is currently Reporter for the American Bar Association Task Force on the Future of Legal Education and has recently served as Chair of the Accreditation Committee of the ABA Section of Legal Education and Admissions to the Bar."

[Not Sure I'm Buying This]

Before I get to my point, I think it's silly to make personal attacks. We're all adults here and should be able to discuss this as adults. In fact, I think Dean Conison should be commended for coming on here to answer David Frakt's assertions.

That being said, I am not buying Charlotte's side of the story. I suppose the most succinct point I can raise is that Charlotte Law's North Carolina bar pass rate declined four yearrs in a row from 78% in 2011 to 65% in 2012 to 60% in 2013 and finally 56% for this year.

This raises a few questions for Dean Conison. First, if Charlotte's admissions program is as effective as claimed, how does Charlotte explain these declines? Does Charlotte see any correlation between declining median LSAT scores and the rapidly declining bar pass rates?

Second, the ABA requires schools to admit students who appear capable of completing law school and passing the bar. Right now, that seems to be around 50% of the starting class (40% fail the bar + at least 10% fail the first year or drop out). Is that an acceptable rate? Finally, if Charlotte feels a 50% failure rate meets the standard, is there a percentage beyond which Charlotte would agree that they are failing to meet the standard?

[anon]

Interesting ...

1. It appears that low LSAT scores are a factor governing the "try it out" program. The notion that students are given an opportunity to actually test their ability in law school courses - a form of conditional admission it seems - seems fair enough. However, two observations: why "
criminal procedure and negotiable instruments"? And, isn't a first year curriculum basically the same thing, albeit a bit more rigorous?

2. The law school has "a very high investment in academic support and bar preparation (e.g., currently 11 faculty and an Assistant Dean solely dedicated to these educational services)" ... this is sort of a shocking admission. If it takes that much effort to coach students to be able to obtain a nearly 50% failure rate on the bar, one can only marvel at a claim of success in this regard.

3. A 145 on the LSAT means that about 75% of the persons who took that exam did better. And that score, as a MEDIAN, is claimed to be acceptable. No wonder so many resources need to be poured into remedial work trying to coach students to pass the bar. Will these resources be available after these students pass the bar, and begin practice? Is it contended that after spending three years in a bar prep course, the 50% chance of passing the bar means qualified attorneys are being trained?

Look, no one would contend that law school should NEVER be available to folks in the bottom 20% of the applicant pool, based on the time honored criteria (GPA, LSAT, life experience overcoming challenges, etc.): perhaps that legal education is available to such applicants should be deemed praiseworthy in certain individual instances.

However, lowering admission standards to basically admit almost anyone to law school is not in compliance with ABA standards, and it is not legitimate to justify taking a lot of money from the vast majority such applicants who will not succeed by passing the bar and obtaining jd required employment.

Where and how should the lines be drawn? This ambiguity permits the "opportunity school" argument, and the author of the post above appears to have been inside the ABA, presumably making it!

One solution suggests itself: measure compliance with vague standards by objective performance in the aggregate. If the majority of a law school's graduates will not be able to obtain the license for which they have studied for so long at such great expense, or, if the majority of students who obtain that license will be unable to find employment requiring it, then the school has failed.

Then again, perhaps the bottom 20% of law schools should be considered "opportunity law schools" for the owners, administration, and faculty, who benefit so richly from their existence.

[Twbb]

"some schools, like Charlotte, utilize more comprehensive, validated predictors of success."

Validated by what? It seems that Charlotte's falling LSAT scores are directly correlated with lowering bar passage rates.

[Kevin Jon Heller]

Dean Conison's first point, about transfers, is completely non-responsive to Frakt's point. No one is claiming, especially not Frakt, that a very low LSAT dooms 100% of students to failure in law school. Of course a few of them exceed expectations and do well. Most, however, do not. The success stories are the ones who transfer; those that do not exceed expectations stay -- and apparently do not do well on the bar exam, if the statistics anon posted are any indication.

[Kevin Jon Heller]

Dean Conison could also settle the matter, of course, by reporting the bar passage rates of students with LSAT scores of 145 and below. If they do not fail at a higher rate than students with higher LSATs, both at Charlotte and at comparable schools, Dean Conison's argument is justified. But I'm willing to wager that's not the case.

[David Frakt]

Thanks for coming to my defense Kevin Jon Heller. See my new post responding to Dean Conison. I don't think it would be enough for the InfiLaw schools to simply report the bar passage rates of students with LSAT scores of 145 and below because these statistics can be highly misleading. Suppose that 4 years ago Charlotte admitted 100 students with an LSAT of 145 or 144. Suppose that eighty of them flunked out, that 20 graduated and that 14 of the 20 passed the bar on the first try. InfiLaw could honestly report that their student from that entering class with a 145 or below had a 70% bar pass rate, even though the chances of graduating and passing the bar for the first try for those 100 students was 14%. That is why I call for data that includes the attrition rate.

[AnonProf]

A shorter version of Conison's argument:
#1 some low LSAT students may be successful
#2 other schools are as bad or worse than us in some areas.

I don't think anyone is arguing against Conison on #1; the real argument, which he sidesteps, is on the probability of success and the disclosure of data around that probability. Frakt's challenge is a good one. Publicly disclose the data for students in the various LSAT bands, with attrition data, so that an incoming student has a good look at their chances. My guess is that 75% or more students with a 142 or below either fail out or fail to pass the bar on the first sitting. If true, that would be good data for all low LSAT students to know. And don't hide behind "disclosing student data," because I have seen a number of schools post claims like 98% of students with an LSAT above 160 passed the bar on the first try. This is asking for a similar disclosure, just for the low LSAT students.

On #2, yes, plenty of law schools are engaging in questionable actions, including the transfer student game. Just because someone else is engaged in wrongdoing does not clear you. I am glad Conison was called out on the transfer point being non-responsive. InfiLaw schools are definitely not the only bad actors, and I imagine very few of those low LSAT transfers are successful at the schools willing to take their money if it does not impact ranking, but InfiLaw schools are probably the worst offenders for reasons Frakt has clearly described.

[anon]

AnonProf

I tend to agree with almost all of you comment, save this: "I imagine very few of those low LSAT transfers are successful at the schools willing to take their money if it does not impact ranking."

Those who have demonstrated aptitude are likely working hard and understanding the material. To be sure, when compared with students who in the aggregate scored far better on the LSAT, students who had a lower LSAT score might not stand out as dramatically. But, please, let's not characterize "success" as the top of the class at an elite law school.

I think everyone would agree that some students, unable to do well on the LSAT for whatever reason, may indeed go on to "succeed" in law school and as attorneys... what is important is to put that success in context, which, as many have pointed out above, Dean Conison does not appear ready to do yet.

[AnonProf]

Anon,

We may just disagree, a bit. I have the LSATs for all the students I teach. Obviously, grading is blind, but I pair LSATs with course grades and names after the fact.

It is certainly possible for students with 142 or below on the LSAT to succeed, but they have some serious challenges to overcome, regardless of the school they attend.

We do not admit students with 142s at my law school, but even the 147-148 LSAT students I teach seem to struggle, even if they work hard. Again, we have had a few lower LSAT students do well. Also, the tests in law school may not measure success after law school.

[David Frakt]

AnonProf -

If your school does not admit 142s how do you know they can succeed? I also taught at two schools where many of the students were in the 147-148 range, and, like you, noted that for many of these students it was a struggle. And there is a huge difference between a 142 and a 147 in terms of aptitude. Here is my assessment of the meaning of LSAT scores and their relationship to risk of failure in law school and on the bar:

156+ High Aptitude/Minimal Risk
153-5 Solid Aptitude/Low Risk
150-152 (44-52%) Reasonable Apt/Modest Risk
147-149 (33-40%) Modest Apt/High Risk
145-146 (26-30%) Marginal Apt/Very High Risk
144 and below (<23%) Poor Apt/Extreme Risk

I should probably add a couple more categories for InfilLaw
141 and below (<15%) Negligible Aptitude/Unconscionably High Risk
138 and below (<10%) Extremely Low Aptitude/Unconscionably High Risk

With respect to transfer students, typically law schools that accept a substantial number of transfer students only admit those who are in the top fifth to top third of their classes. These students generally have the ability and work ethic to succeed no matter what school they are coming from and tend to do quite well at the schools that they transfer to.

[AnonProf]

I just mean it is theoretically possible, but I agree with you, their chances are extremely slim and schools, for the most part, should not be admitting at that level.

[Rob]

The AAMPLE program is being used effectively at INFILAW and NSU. This "admissions program by which students with lower LSAT and UGPA enroll in two challenging law school equivalent courses" is licensed from NSU. Dean Harbaugh of NSU recently received a patent on this process: http://nsunews.nova.edu/beyond-the-lsat-nova-southeastern-university-law-professor-receives-patent-for-an-alternative-admission-model-program-for-legal-education/

I suspect AAMPLE will be adopted by many more law schools as the shake down in legal education continues.