Adding Clarity to Law School Transparency

Lower tiered law schools that admit students with low LSAT scores should be celebrated for providing an opportunity to enter the legal profession for those students at the margins of society.  Unfortunately, their work has lately been characterized as predatory in popular media like The Atlantic and the NYT.  With its 2015 State of Legal Education report, Law School Transparency (LST) has instigated a dangerous national discourse, arguing that some ABA-accredited law schools have been engaged in a process of admitting students with low LSAT scores (which the LST labels “high risk”) who do not have a reasonable chance of passing the bar.  LST’s framing of these law school’s actions is disturbing given that: (1) the LSAT is at best a weak predictor of first-time bar passage, and (2) there is no evidence that law school graduates are not eventually passing the bar.

 

As a sociologist and law professor at Whittier Law School—one of the law schools that LST has labelled as high-risk—I decided to take a closer look at the results that LST relies on in coming to its conclusions about bar passage and risk.  LST relies strongly on results published in the LSAC National Longitudinal Bar Passage Study (LSAC Study).  I agree with the recent press release by the LSAC that it is problematic to use a 25 year old study to assess current bar passage risk.  Over the past 25 years, the LSAT has not only changed with respect to its score scaling, but also its content and substance.  However, even if you assume you can draw current inferences about today’s LSAT from the decades old LSAC Study, it fails to provide evidence supporting LST’s contentions.

 

LST clings to the false premise that the LSAT has a strong correlation to bar passage. 

This presumption is necessary in order to maintain a system of classification that categorizes a student’s risk of failing the bar based on her LSAT score.  Citing the LSAC Study, the LST 2015 State of Legal Education report overview, states that the LSAT is the best predictor before law school as to whether a student will pass or fail the bar exam.  But this statement is a bit misleading for two reasons.  First, it does not acknowledge that the indicators that do a better job predicting bar passage are those that appear after law school begins—i.e., a student’s law school GPA.  The LSAC Study states clearly that—of all the variables used in the study—cumulative law school GPA is the best predictor of first time bar passage.  This should come as no surprise.  What a student actually learns in law school should have significant salience in her first-time bar passage.

However, LST focuses on admissions standards.  Which leads to my second point as to why the LST’s statement is misleading.  The LSAC Study never stated that the LSAT is the best predictor of bar passage.  It states that of the factors that it has pre-selected, the LSAT is the best predictor of first-time bar passage.  It is important to state that the LSAC Study considered a few “before law school” variables in addition to the LSAT (for example, undergraduate GPA, college selectivity, law school status, socio-economic status (as measured by family income in high school and parents’ occupation), and race & ethnicity).  One could easily imagine that a factor like a student’s family income or financial resources available (a) when entering law school, (b) during law school, or (3) in the three months prior to sitting for the bar could be extremely probable and plausible factors that might not only correlate, but might actually cause some change in student performance with respect to first-time bar passage. It is vital to point out that the idea that the LSAT is a best indicator is relative, and in fact the LSAT might only be the best among a limited number of selected variables.

What LST fails to report is that while it is true that the LSAC Study reports that the LSAT is has a statistically significant correlation to bar passage, this correlation is weak at .30.  The LSAT as a single variable explains less than 10 percent of the variation in bar passage.  Only when the LSAT is used in a model with law school GPA does the correlation get stronger—and even then it is somewhat weak.  A majority of the variance (about 68%) in bar passage outcomes cannot be explained!  This means that how people actually perform on the bar varies a lot from how one might expect them to perform on the bar given their law school GPA and LSAT score.  We clearly need other factors to explain this variance.  In order to do this, it might make more sense to examine what happens to the bar taker after law school begins but before sitting for the bar—particularly exploring how students are preparing for the bar,  and analyzing their material and economic resources during prep period.

Instead LST’s overemphasis on the LSAT, treats the weak correlation between the LSAT and bar passage as somehow causally connected.  Granted, LST has never explicitly stated that there is a causal connection.  To their credit, they have stated that there are things, like increases in undergraduate GPA, that could offset a low LSAT score.  However, in the absence of this offset, LST treats the LSAT as having causal power—the LSAT either needs to be offset with higher undergraduate GPA or individuals with low LSAT scores and average uGPAs should be denied admission.  Even the authors of the LSAC Study warned against this crude thinking, and stated explicitly that the presence of a correlation does not mean that raising law school GPA or LSAT scores will lead to an increase in bar passage.

 

LST Overemphasizes First Time Bar Passage

This is less an issue of empirical debate and more of an interpretative discussion.  LST states that law schools are accepting unqualified students who can’t pass the bar, yet they offer no evidence that these students are not passing the bar on subsequent attempts.  You cannot solely measure passing the bar on whether students pass the bar on the first attempt.  That is unacceptable.

Of course every school should strive to increase their first time bar passage rate, for the sheer reason that multiple bar attempts can create economic and psychological hardships on graduates.  However, we should not overemphasize first-time bar passage so much that we ignore the fact that there are many graduates who pass the bar on subsequent attempts and enter a legal profession that either requires their bar standing and/or welcomes the advantage that their JD training provides.

According to the LSAC Study, 98% of bar takers with an LSAT score at or above the mean eventually passed the bar, compared to 90% of bar takers with an LSAT score below the LSAT mean.  There is likely evidence that eventual pass rates are well above 75% for low tiered law schools.  We know that one of the ways in which law schools with low first-time bar passage maintain their accreditation under the ABA is because they maintain a bar passage rate above 75% for those students who have graduated in the most recent five year period.  Eventually passing, whether on a second or third try—or within 5 years of graduation—seems like a far more important outcome than first-time bar passage, if we are seriously concerned with whether graduates are entering the legal profession.

 

Law school administrators and faculty must resist perpetuating and participating in the dangerous narrative advanced by LST.  Others have highlighted the paternalistic and problematic nature of this narrative.  In recent years, law school applications have been in decline.  This has made it somewhat easier for an individual to get accepted into a law school.  When admissions standards become easier, those at the margins of society—specifically the poor and black & brown racial and ethnic minorities—have the potential to benefit.  But just at the moment when it becomes easier for those at the margins to gain admission into the legal profession, the media and organizations like LST engage in a crusade to vilify law schools and pressure them into accepting fewer students.  This is done in the name of “transparency” and a paternalist drive to help the “unqualified” applicant who is at high risk of failing the bar.  Instead, LST’s strategy is at high risk of keeping black and brown people, and the poor from having the opportunity to ever sit for a bar. 

Comments

[Leo]

Whittier and schools in its class would not give the students they are now admitting the time of day when things were going great guns a few years ago. Now they are the Mother Theresa of Legal Education for giving these poor souls "opportunity."

What I want to know is this: Sheldon do you really believe this BS??

[randomanon]

Opportunity is important, I think. But I think there's also a question of realism and tradeoffs. Realism in terms of the state of the legal market, and tradeoffs in terms of debt levels. And simply labeling as 'paternalism' any concern for the debt levels proved to be an unavailing argument.

If Prof Lyke really believes in empiric methods, he would not have written this post. There is a paucity of data out there, and no data set is perfect. But faulting any results that derive from an imperfect data set does not advance the conversation.

I wonder if it wouldn't be helpful to collectively brainstorm on what data would be useful to assemble to get an accurate picture of things.

Various methods have been employed, all of which are useful-ish attempts but inadequate in various ways.

- econometric modeling with limited and outdated data sets with no consideration of debt levels or repayment
- D Merritt tracking down one state's bar passers a few years out
- Looking at bar passage/starting salaries at the very beginning of a career
- Looking at a severe drop in incoming LSATs and beginning to see the corollary drop in bar passage

LST has said that "the available data confirm the validity of the risk categories as presented in the report. Again, LST invites any law schools with data tending to refute the validity of these categories to come forward and make that information publicly available. We want to be wrong."

I suggest that more data from the gov't about IBR/PAYE for law grads would be very useful to see.

Law schools probably have a decent amount of info for all grads of at least recent years that could make a good data set. I'm assuming LSAT, GPA of ugrad and law school, bar passage/attempts, debt level, whether or not they enrolled in IBR/PAYE (not positive law schools know this, but they might), type of job grads are in. Some of this info gets reported separately, but getting all of this info reported together would be informative. A uniform format.

Who/how to induce schools to share this info is another matter. And privacy is also a concern, but a manageable one. LST? The ABA? And how to get law schools to play ball in providing yet more data. And a range of schools in terms of quality.

Law schools have contact info for all recent alums, I'm sure. Some social science researcher should put together a study/survey and partner with schools to access their alum lists. Would need to be short and simple enough to get some reasonable response rate but also informative.

But what data would make an empiricist happy or would be the most meaningful?

[MT]

Empiricism aside, no further data is needed to expose the author's attempt to stake out a moral high ground. The author claims - presumably with a straight face - that low tier law schools such as Whittier should be "celebrated" for accepting millions of taxpayer dollars each year in order to hand over law degrees to basically anyone who can fill out a federal loan application. That's may be a fantastic business model, but it's basically the opposite of altruism.

A few years ago, the founders of very low tier Charleston Law School withdrew $25 million in profits. Until a couple years ago, the dean of the low tier New England School of law took home $867,000 a year. And on and on. But those schools provided "opportunities" to many students with low LSAT scores, so apparently they should be celebrated.

[California guy]

We had an immigration attorney here in CA who would lie to clients, saying he had connections in high places. He would charge them several times what other practitioners asked for, explaining that he needs to the money to grease the right palms. These clients, coming from cultures where corruption is commonplaces, bought into his lies. This attorney maintained a collection of luxury cars in a large garage on his property. He would do nothing for these people, dozens of them, while lying to them the whole time that he instituted administrative and court proceedings to get them citizenship and/or stop their removal.

He represented himself at his State Bar Court trial. In his defense, he testified that it was unfair to disbar him because his clients were mostly poor minorities. He stated that disbarment would be prejudicial to these communities, because they were underserved by other lawyers.

He was disbarred anyway.

[Sy Ablelman]

Diversity in legal education and the profession is needed. However, even if law schools accepted students at the "margins" to pursue their "dreams," it does them no damn good if there are no JOBS! Governments are NOT hiring attorneys and prosecutors. I was in a traffic court today with at least 300 defendants on the call, many with serious matters such as DUI. There was exactly 1 prosecutor. Public service jobs and small firms jobs are just no there. Hanging a shingle is not an option unless you are connected, have an established referral network and clients. These new kids are competing with folks like my buddies and I, all unemployed and under employed lawyers out many years. It is not a celebration to graduate these kids into this market.

[MrNiceGuy]

@Boby:

"1) Many students don't pay such tuition because they receive scholarships (or so I assume).

2) Most of the students pass the bar eventually--albeit the second time.

3) Many great students with low LSAT transfer after the first year.

4) Whittier, it seems, is one of the most diverse law schools in the nation. In California, where only 4.2 percent of Latinos are members of the bar--low tier law schools play an important role."

Re (1), I assume that you assume wrongly. How is speculation helpful here?

Re (2), others have cited evidence to the contrary. Are you assuming again?

Re (3), stats or it didn't happen.

Re (4), how does Whittier "seem" that way? Do you have numbers, or are you speculating? Since assuming things is fun, let's assume you're right that the student body at Whittier is impressively diverse. How many of the "diverse" students are passing the bar, entering the legal profession, or otherwise gaining anything apart from crushing debt from their experiences at Whittier?

Others have offered statistics to the effect that the outcomes for most Whittier students are dismally bad. Would you care to offer some assumptions to the contrary?

[Ex Cardozo late 80s]

Ditto Cardozo Law. I attended late 80s and the school stats were glowing how many grads were "employed" and they would market themselves by showing a "representative" sample of grads working in top law firms making the "going rate" which at the time was approx 95k or 110k for 1st years. What a great PR dept because ALL my friends and close classmates and that means a large majority of the class were employed post Cardozo in...DA Office bronx, bklyn, queens, or or Legal Aid, or went to work for PI solos or small law offices. Many could not find a job so they went to work for family law firms ... the small practice of a father or other relative. The return on tuition was very bad. Some could not find any law job as they were unconnected so they became involved with non legal jobs. But the Cardozo alumni mags kept showing off the very few who got in either by being on Law Review or Connectipns into a large rich well paying law firm. Over the years I have been in touch with several of my friends and they are either struggling badly and still hoping for thay big PI case or just barely making it.
I have no idea if it has improved although by inference the Cardozo grads that I have met over the last 10 years are all in the same boat - ie solos or working for the DA or small law offices. I am sure some have "made it" but the risk reward ratio is very bad.