Is Law School Transparency a Reliable Source for Information?

Did you know that Kansas Law charges $21,173 for in-state, resident tuition?  That Iowa Law charges $51,864 for non-resident tuition?

Did you know that the interest rates for graduate federal direct loans are 7.1% and 8.1%?

Well I hope you don’t know any of that, because all of that is grossly inaccurate.  In fact, here at Kansas we posted our tuition rate of $19,985 in July 2014.  Iowa Law has made available its $41,296 annual non-resident tuition available for a similar time period.   As for the federal direct loan interest rates, the Department of Education posted the correct interest rates for student loans way back in May 2014 (between 5.41% and 7.21% depending on when the first disbursement occurred).  So you would think a webpage that estimates how much it costs to attend law school for the Class of 2013 would use these published figures.  Well, despite this data being freely available and very easy to find, Law School Transparency does not use this information when calculating their cost of attendance estimates.  Instead of using accurate, available data, they rely on projected estimates for law school tuition and for federal loan interest rates, which is another way of saying they’re using guesstimated data instead of real, available data.  As a consequence, their estimate assumes Iowa students are paying $51,864 when we know they are paying $41,296.  Not surprisingly, this error causes LST to significantly over-estimate the cost of law school.  This is not specific to the entry for Kansas or Iowa, they have used the same method for all 200+ law school entries on their website.

I understand that if you dig deep into the webpage, you can see what LST was doing here.  On the Iowa “costs” webpage which is tailored to the Class of 2013, when they say “Non-resident Tuition: $51,864”, they don’t actually mean that’s the tuition for U. of Iowa for the Class of 2013.  If you carefully move your cursor to the very tiny and easy to miss “?”, you will see that that figure is an estimate based on the prior year’s tuition in relation to tuition growth during the five years prior to the current year (2008-2013 instead of the more timely 2009-2014).  Took me a number of visits to the LST website before I figured out that “tuition” doesn’t necessarily mean tuition on the LST webpage, it often means a guess at what tuition might be.   Which, again, is a bit strange considering the actual data is freely available.  Isn’t this the same kind of sleight of hand that LST has accused law schools of doing?

Listen, I like Law School Transparency.  I think they’ve been effective pushing law schools to be more accurate and transparent in their reporting, and that’s a good thing.  And I suspect this is a product of a bungled, confusing webpage and a failure to make timely updates, rather than some devious plan to mislead the public.  But the website does mislead its readers into thinking the cost of law school is higher than it actually is.   I think it’s fair to expect LST to hold itself to the same standards it expects of law schools.  If law schools were reporting estimated LSAT and GPA medians instead of using actual available data, I think LST would be justified in strongly criticizing law schools for the practice.   And vice versa.

LST presumably knows these numbers are wrong, which raises the question as to what is LST’s duty to correct the record?   As Ben Barros posted last week, followers of LST organized to update all 200+ ABA accredited law schools’ wikipedia entries with tuition and employment information based on LST’s faulty numbers.   Will LST correct its own website?  Will it encourage its followers to correct the Wikipedia entries?  Will it contact journalists who have published inaccurate data based on the LST data?

Update:  I encourage readers to scroll down to Kyle McEntee's response in the comments section, which got caught in the spam filter.  I'd also like to second Bernie Burk's thumbs up for LST's new podcast series.  I've only listened to the first one, but it sounds like a great series.

Update #2:  Kyle mentions he's been trying to get updated tuition and cost data from the ABA.   In December, the ABA published every school's Standard 509 reports, which includes the relevant data for tuition, fees and cost of attendance.  You can find it at http://www.abarequireddisclosures.org.   

Update #3:  Mack, I have removed the posts related to the spam filter because they are off-topic and I have nothing to do with the settings on the spam filter.

Update #4: Several commenters have suggested I should have contacted LST about this issue prior to this post.  I did raise this issue more than two weeks ago in the comments section to Ben Barros' post.  Kyle McEntee responded he would update when new data became available, but that he would not update or correct any previously posted erroneous data.  Despite having been notified that the interest rates were inaccurate, he did not make a correction or notification on the LST website.  That exchange is what prompted this post.  You can review the exchange at http://www.thefacultylounge.org/2015/01/law-school-transparency-wikipedia-pages-and-law-school-debt.html

 

 

 

 

Comments

[KeepSake]

Another big problem with LST is that it counts employer funded jobs in its LST score. It also fails to account for school funded jobs in its underemployment score. (The latter does not even come with an asterix). By doing this, LST is encourging students to choose schools such as George Washington, William and Mary or Emory which have perhaps 20 percent of its class in such jobs, but are not doing a particularly strong job of encouraging actual employers to hire their students. The LST score and underemployment score would be a lot more useful to applicants if they did not include these jobs. (And I know, LST will say if an applicant takes the trouble to figure it out the information is there, but when you have an overall score that is what most applicants will focus on).

[JusticeFor All]

Don't let the perfect be the enemy of the good. LST provides an invaluable resource for prospective law students. If law schools reported anything close to accurate statistics for its graduates in terms of employment and salary, LST would become irrelevant. Law schools don't do that, therefore LST exists. With regard to Wikipedia pages, schools game those too with highly misleading information. For example, the University of San Diego school of Law's wikipedia page lists the following:

Students[edit]
In 2007, the 75th percentile of Law School Admission Test (LSAT) scores was 164 and the 25th percentile of LSAT scores was 161 for admitted full-time students.[13]

Bar passage rates[edit]
Based on a 2001–2007 6 year average, 76.3% of USD Law graduates passed the California State Bar.[14]

Post-graduation employment[edit]
Based on a 2001–2007 6 year average, 90.6% of USD Law graduates were employed 9 months after graduation.[14].

Essentially touting the wild-west, report your own employment numbers without audit or verification and stopping at 2007, which is conveniently the year before the recession hit.

[Stan]

Is KU's own website a reliable source of information, Steve? You barely address the employment outcomes of your students (25, 50, 75 percentiles is all I could find). A far cry from other public school systems such as Michigan.

Perhaps you should think carefully about the type of house that you happen to live in before you cast any stones?

[terry malloy]

So, what was KU out of state tuition in 2013? $49,025, right? (funny how that fact doesn't make it into your diatribe).

Their projection was correct at the time it was made.

Updating analysis, data, and websites takes time and money.

If you want LST to update their data more frequently, you should donate.

[Nathan A]

We could have used this commitment to accurate reporting when law schools were publishing misleading and highly dubious employment outcomes for their graduates.

But yes, LST should correct those errors and I have no reason to doubt that they will. As you say - "I suspect this is a product of a bungled, confusing webpage and a failure to make timely updates, rather than some devious plan to mislead the public." It not like they're making the kind of money law school administrators make for providing this service. Try to relax.

Also, I recommend having a little more faith in people's Google skills. A prospective student inquisitive enough to find LST or look a Wikipedia entry is probably inquisitive enough to check a school's website. Really.

[jeo]

So, it was accurate when posted (and it clearly states at the top of the page "CLASS OF 2013"), and then KU reduced tuition for the 2014-2015 school year? Sounds like it is accurate information and will be updated for 2014 when those annual numbers are available.

[Jojo]

Kyle,

"First they ignore you. Then they laugh at you. Then they attack you. Then you win." Keep up the good work. Transparency harms no one but the scoundrels. Your critics do not care about transparency or their students; they care about their jobs and their pocketbooks. Keep up the good work.

[Kyle McEntee]

Keepsake,

Unlike other law school critics, we do not have organizational opposition to school-funded jobs. Indeed, some of the jobs that count in the Employment Score are a joke and a naked attempt at boosting employment rates and rankings. But others are not. UVA's program, for example, has their graduates in school-funded jobs in better jobs making more money than a lot of other people in so-called real jobs. We believe that the asterisk is a nice spot in the middle where we alert readers to the inclusion of school-funded jobs so that they can make their own judgment.

As for the Under-Employment Score, it does account for school-funded jobs that are either PT or ST. If we were to do an asterisk there, it would be to signal that certain school-funded jobs (LT, FT BPR) did not get swept into this number. On balance, I think that would be more confusing than not, so we left it off. And again, we do not have an organizational opposition to these school-funded programs.

[jeo]

Just looking at the LST page again for Iowa Law. The top of the page says:

Published Cost of Attendance 2013 - 2014 Academic Year | Non-Resident Tuition & Fees: $49,025

Then below that it says:

Hypothetical Price Discount Tables | Non-Resident Tuition: $51,864

What's confusing about this?

[Kyle McEntee]

Looks like my reply to the OP got caught in the spam filter. How frustrating! The captcha on the site really should be enough to catch spam. I for one would rather wade through minimal spam to have a better, free-flowing conversation.

[John Steele]

"And why beholdest thou the mote that is in thy brother's eye, but considerest not the beam that is in thine own eye?"

And to answer the question directly, "Yes. Yes it is."

[jeo]

From the Iowa Law web site in 2011:

"Historically, 99% of Iowa Law graduates are employed within nine months of graduation."

Reality:

Total 2010 Graduates: 212

This school did not hire any of its graduates.
68.4% of graduates were known employed in full-time legal jobs. This figure includes an unknown number of temporary jobs.
75% graduates were employed in long-term jobs.
81.1% graduates were employed in full-time jobs.
0% of all graduates secured job offers by graduation.
0% of all graduates obtained their jobs at Fall OCI.

[MacK]

"Looks like my reply to the OP got caught in the spam filter. How frustrating! The captcha on the site really should be enough to catch spam. I for one would rather wade through minimal spam to have a better, free-flowing conversation."

Kyle, I have long noted that it is curious what gets caught in this "spam filter" and what does not. It seems that certain critics of the positions of leaders on this blog get caught a lot - and sometimes the caught posts steer the debate in directions that seem to suit some people. Other posts get mysteriously delayed, sometimes for a day or two, so that they appear way up thread. All very curious.

Let's see if this post gets posted.

[Kyle McEntee]

Steve,

I'm always happy to discuss our site and our methodology.

Before I get into this, I want to correct an inaccurate statement you've made. No reasonable reader is going to think that KU or Iowa's tuition rate is 21k or 52k. It's in a section called "Hypothetical," labeled "price starting points," and explicitly defined as 2013-14 tuition plus the tuition inflation calculated from the previous five years of tuition increases at KU specifically.

A few years ago, we collected self-reported tuition/fee/cost of living data from every law school. It turns out that the information reported by more than a handful of schools was grossly inaccurate, so we decided to wait on the data from the ABA from now on. It's the best way to get accurate information from schools. That accurate information, along with historical tuition hikes, is the basis by which we make what's very clearly projections.

Somebody from the ABA staff can confirm that I have been hounding them by phone and email every week for updated tuition/fee/cost of living data. I know they're working diligently, and that they know I want data yesterday, but their staff limitations are real and I understand that.

When we update the cost of attendance data, we will also update the projections, which will necessitate us projecting three separate interest rates, along with an implied projection that the method of calculation will remain the same for the covered years (a legislative projection). As we do now, we will make the assumptions clear to readers in a number of places. First, on the costs page of every school profile. Second, on the guides. Third, on the project page that identifies the assumptions used for the projections.

I expect our new projections will be available in the next two weeks or so. When this happens, we will necessarily be making assumptions about tuition, cost of living, and interest rates over the next few years. When we get data we trust from the ABA for all schools calculated in the way the ABA asks, we will update the projections again. What we don't want to do, however, is make changes to the projections after the fact because I think that would be confusing for users. This is a trade-off, to be sure, but one I am confident makes sense in light of all of the moving parts.

Two final points about the assumptions. First, the interest rate for 2014-15 may not have been right, but we have no idea if we got it right or wrong for 2015-16 or 2016-17. We might have undershot it for those years, or overshot it. Either way, predicting interest rates that far out is an unknowable task. But we're clear that they're predictions. Second, the tuition and fees we use for 2014-15 may have been a little low, but we don't know what the amount will be for 2015-16 or 2016-17. Again, we might have underestimated it. There are many examples of law schools that kept tuition flatter from Y1 to Y2, but from Y2 to Y3 and Y3 to Y4 increased it much higher. When you're using historical increases to project forward, there is some natural smoothing out. We think it's important to show this to users, so we do.

One last note. Users can and do use our custom cost of attendance tools to update the data for their own projections. They can set interest rates, very specific living expenses, and tuition for the years they will be there. When this is there, the custom reports, school profiles, and school comparison page all include their tailored numbers.

But don't just take my word for it. I invite readers to judge for themselves whether the information is posted in an inaccurate or misleading manner. Here are links to the Costs tab for KU, as well as a screen cap indicating the relevant portions of the page:

(add http to the front of each to make it a link -- I don't want to be caught in the spam filter for linking)

lstscorereports.com/schools/kansas/costs/2013/

i.imgur.com/JvXMWcG.png

[terry malloy]

It is amusing that the success of LST in informing consumers of legal education led to the collapse of applicants, which spurred the tuition discounting that the Baghdad Bob Admissions Dean is apoplectic about LST not yet updating.

also: "We use ABA numbers because your peers have lied to us in the past."

Point Kyle.

[Michael Risch]

Mack -

You can probably put your spam conspiracy theory to bed. My own comments, while logged in to typepad, responding to comments ON MY OWN BLOG POST get caught in the spam filter. If stuff shows up later or never, it is based only on whether someone actually looked and also had permission to release, which appears to be limited.

[No, breh.]

ITT: "sophisticated consumer" fails to read web page, nonetheless offers criticism.

[Kyle McEntee]

Re: update #2. The 509 reports are available, but require mining through all 200 reports and manually entering multiple data points. We do not have the resources to justify the many, many hours that would take to do. Our QC for all manual data entry requires two sets of eyes.

When the ABA tells me they will send me a spreadsheet with the cost data (and a lot of other data) very soon, I cannot justify spending my time on that. That it's been pushed back months is unfortunate. If you would like to find student volunteers to do data entry in a timely fashion, please do so and I will coordinate with them to get the data entry completed through our QC process.

[Steven Freedman]

@ Kyle and others...

Posters have asked what's confusing about the "hypothetical price discount tables"? There's a lot to be confused about with this table, but let me just pick two items, one of which I've already mentioned:

* When you hover on a dollar figure on the chart, it says "ten year plan" and "twenty year plan". What ten year plan? What twenty year plan? I'm particularly curious about the twenty year plan. The only 20 year plan I'm familiar with is PAYE. But if LST is referring to PAYE on the chart, then the monthly payments would be much lower than indicated. Also, PAYE provides loan forgiveness after 20 years. Is this reflected in the 20 year plan data? Or is LST confusing the 20 year plan with the 25 year extended repayment plan available to students?

What's most confusing is that Iowa's tuition for this year is $41,296, data that's been public since last summer. So why are the calculations based on a starting point of $51,864? Also confusing, the federal direct lender loan rates range from 5.41% to 7.21%, yet the estimates are based on 7.1% and 8.1%.

Why use "hypothetical" numbers, when real data is available?

[Jojo]

Why would anyone write a hit piece on Law School Transparency? Their data are objectively not confusing, and certainly more accurate than the data massaged by various educational institutions from a few short years ago. They don't have the conflict that schools do, notwithstanding an earlier hit piece that tried to depict Kyle as an extortionist.

LST is a good and useful organization. Assuming your sincerity when you say that you "I like Law School Transparency" (and I have my doubts as to the veracity of that statement given your earlier criticisms of LST), the proper response is to e-mail Kyle McEntee with a question or with your school's updated data. He seems to be pretty reasonable and dedicated to accuracy here. You could even make a donation to LST if you really do "like Law School Transparency" as you purport to do. Alternatively, you could come clean and volunteer that you loathe LST and view it as a threat to entrenched interests.