Heat vs. Light II: It Matters Who Counts and Who Cares

This is the second in what I intend to be a short series of posts on the importance and limitations of Mike Simkovic’s and Frank McIntyre’s recent contributions to the literature on the purpose and value of a legal education.  (The first is here.)

I’ll devote this post to setting aside what I think is a peripheral issue that has occupied some of the resurgent debate spawned by Mike Simkovic’s series of posts on Brian Leiter’s blog.  In recent exchanges, this issue has emerged as a series of accusations and assumptions about what employment and earnings data ought generally to be presented, and whether the way such data are generally understood makes their presentation subject to misinterpretation.  A lot of the recent controversy has focused on whether, when a law school reports its graduates’ employment numbers, “employed” should or should not include part-time, short-term or non-law-related employment; and whether “unemployed” should include only those actively seeking work, excluding the category the ABA collects as “unemployed—not seeking.”  Similar disagreements persist over how to count school-funded positions, and what qualifies as law-related employment, though these are not front-and-center in the recent exchanges I’m writing about here.

Mike’s recent posts have taken the strong and categorical view that law schools, NALP and the ABA ought to report law-graduate employment the same way the U.S. government reports on employment generally, and that any other view is ignorant or misinformed.  Board of Labor Statistics and Census data (among others) report people as “employed” if they have any kind of work at all, including work that is part-time, short-term, or (in the case of law-school graduates) entirely unrelated to their legal education; and as "unemployed" only those who are actively looking for work.  The widely articulated criticism “that law schools behaved unethically or even committed fraud . . . by presenting their employment statistics in a misleading way,” says Simkovic, “comes down to this:  The law schools used the same standard method of reporting data as the U.S. Government.”  Any contrary view, he argues, is “based on an incorrect belief that law school only benefits the subset of graduates who practice law . . . ,” a point he not infrequently expresses as an impatience for those who would be influenced by near-term results resembling the ones they might desire for themselves rather than more dispassionately considering lifelong outcomes of the JD population as a whole to assess their prospects.

Not so fast.  It matters who counts, and who cares.  As for who counts, there is a very significant issue in equating the accuracy and reliability of statistics gathered and disseminated by disinterested government employees with those gathered and disseminated by self-reporting law schools, especially in an environment in which both rankings and prospective law student decisions essential to the welfare of the reporting school are often based on what it reports.   I have no doubt that many schools are meticulous and unstintingly honest in their self-reporting.  But we already have several documented incidents of motivated or flatly dishonest reporting, and my regretful suspicion is that such discrepancies will prove to be more common than anyone would hope as the ABA starts auditing.

As for who cares, my modest proposal is that people should get the information they want in the terms they are most likely to understand.  I would guess that a libertarian economist like my friend Mike Simkovic agrees, in which case his arguments on these issues don’t square with our apparently shared principles in at least two important respects. 

First, with respect to the proposition that people should get information in the form they are most likely to understand:  It is demeaning and more importantly inaccurate to argue that everyone understands—or any “reasonable” person should—that law schools reported employment statistics the same way the government does.  In fact many if not most people relying on US News rankings and law-school statistics (among others, tens of thousands of prospective law students and their families each year) didn’t understand that at all, which is why the ABA stepped in to change the reporting requirements in 2010.  I personally was stunned when I first learned that law schools’ “employment” rates reported before 2011 included part-time, short-term and non-law-related work.  I am not, as I have previously cautioned all of you, a social scientist, but I daresay I’m no more stupid or ignorant than the next guy.  Countless others have expressed the same surprise, and even greater surprise at the discrepancies between “employment” as reported before 2011 and the more granular information reported afterwards.  We needed more detailed and explicit reporting for the same reason that just about every state has consumer-protection legislation governing the form and content of disclosures regarding common consumer transactions:  You take your audience as you find them. 

Second, with respect to giving people the information they want:  Simkovic and McIntyre are free to argue (as they do) that any prospective law student’s interest in the nature or duration of near-term post-graduate employment is “based on an incorrect belief that law school only benefits the subset of graduates who practice law . . . .” And they have in fact presented significant empirical evidence suggesting that it is possible for some people to overemphasize these near-term details at the expense of best-informed decisionmaking.  As I will discuss in future posts, that doesn’t mean that these conclusions should immediately be uncritically accepted as immutable laws of the universe, and it especially doesn’t mean that they are true for all of the people all of the time.  And I assume that, even if they are fully convinced of the truth and immutability of their conclusions (a conviction I suggest is premature), they are not arguing we should withhold information from people so that they can make decisions that the authors consider “better.”

Just as importantly, Simkovic and McIntyre’s studies attempt to quantify the effect of a JD degree on career earnings, and offer suggested courses of action some might choose to take if their goal is to make as much money as they can over a lifetime.  No one could claim that this is an irrelevant perspective, but it is hardly the only one we see:  While it certainly isn’t true of everyone, many people actually go to law school because they want to be lawyers.  Some crave the drama of the courtroom; some have specific social or political purposes in mind; some (and there is at least one study suggesting that this may be particularly true of some who attend lower-ranked law schools with less robust employment prospects and are among the first in their families to seek higher education) enjoy satisfaction from the prestige of being a professional.  Even the most doctrinaire neoclassical economist would not presume to argue with a consumer’s utility preferences—which is just econo-speak for there’s no accounting for taste.  For the population—and I will suggest that it is a significant one—that specifically wants to be lawyers, statistics about any old job held by JDs are not very helpful. And if you’re still thinking that this doesn’t reflect the real thoughts of real people in the real world, consult the recent exhortation posted by the dean of Cooley Law School—an institution that, with all respect, has an uninspiring recent track record in placing its graduates in lawyer jobs—proclaiming “Now Is the Time to Fulfill Your Dream of Becoming a Lawyer!”

So let’s talk about what information people might want, and let’s talk about how they ought to consider using it.  I think Mike Simkovic’s recent posts on the Leiter Blog focus predominantly on just that, and hats off to him for doing so.  Let the discussion continue. 

But I do want to close by taking issue with one of those posts, which I view as a cautionary tale about the temptations of partisanship.  On March 31, a Berkeley Law professor published an op-ed in the New York Times entitled “Law Schools and Industry Show Signs of Life, Despite Forecasts of Doom.”  The first paragraph poses the question “is now a good time to go to law school?” and the balance of the op-ed argues hell yes, focusing on (among other things) the employment prospects of recent graduates of Georgetown Law, and on the growth of “quite profitable” large law firms and in-house law departments to argue future need for “lawyers, and plenty of them.”  Kyle McEntee (of Law School Transparency) and Steven Harper (of Belly of the Beast) both took strong issue with these arguments on a number of grounds, principally that they were inexcusably sloppy and cherry-picked statistics misleadingly—for example choosing one of the top law schools’ employment outcomes for an example when scores of them have much worse results; suggesting that large firms are hiring aggressively when in fact they are hiring about 30% fewer new lawyers than they did seven years ago; and using part-time, short-term and non-law-related employment outcomes to argue that we could anticipate lots of future demand for lawyers.

One of Mike’s posts not only defended the op-ed uncritically, but did so in terms and in tone that (I hope he’ll forgive me for saying) fall well below the standards I have come to know him for.  “Mr. McEntee’s problem,” Mike wrote, “is not that The New York Times got the facts wrong.  His problem is that The New York Times got too many of the facts right.  Mr. McEntee simply dislikes the facts.”  With all respect to Mike Simkovic, that’s really quite unfair.  In my own view, McEntee and Harper both nailed it.  The op-ed author used cherry-picked statistics including non-lawyer employment and lawyer underemployment to predict (among other things) the future demand for lawyers.  That’s misleading.  And the entire thrust of the op-ed—“is now a good time to go to law school?” and can those who apply to law school now expect to work as lawyers?—is antithetical to Simkovic’s and McIntyre’s hypotheses, which are that it’s a bad idea to ask whether now is a good time to go to law school (because you can’t know, and any one time is as a matter of prediction as good as any other), and an even worse idea to worry about whether you’re going to be a lawyer when you’re done (because that could distract you from the long-term earnings enhancements that they believe JDs offer most comers).  Mike should have lambasted the editorial for citing his work while missing his point.

Next time we’ll talk more about Mike’s points, including both their valuable insights, and those insights’ potential limits.

--Bernie

 

Comments

[Barry]

"The moral critique against law schools comes down to this: The law schools used the same standard method of reporting data as the U.S. Government."

The fact that the US government does something does not mean that it's ethical. And in this case the defenders of legal academia were using definitions which suited them just fine, and were against the best interests of potential students.

[Anon]

Barry writes:

"McIntyre is not a law professor, and might have an academic reputation at stake[1]. He might be trying to disassociate himself from this."

The absurdity of this on its face is one more reason why this site has deteriorated so badly under the assault by mindless law school critics. McIntyre co-authored the second major paper on the topic with Simkovic called Timing Law School which was only recently released. Frankly, Simkovic is doing this crowd a favor to waste his own precious research time trying to correct all the mistakes you people make.

[anon]

Anon

SOrry you are so upset by "this crowd," but one could legitimately wonder the reason that S is so involved in so heavily promoting the sort of self-evident set of propositions he has proposed, i.e., that after investing 200K or more, three years of time, and substantial intellectual capital "most" law school graduates have historically found that investment conferred some benefit in lifetime earnings, and 2.) timing the entry into graduate school may affect lifetime earnings, but, over a hypothetical 40 year period of working, historically, these differences have evened out.

One may wonder: why would a law prof with S's education, training and employment experience be so involved in such substantial efforts in blogging about these topics, taking on "all comers" with what is charitably characterized above as "impatience" (unlike Anon, who just starts biting and spitting at the drop of a hat)?

A sincere academic interest and desire to combat what is perceived as the "mistakes" all "you people" make, it appears.

I don't hold S completely responsible for the wild claims that are made using the papers he has co authored with M. And, I think M has a role that is more directed to supplying some expertise about labor economics, not blogging and arguing with everyone about all this.

What is the reason that such vehemence is invested in such anodyne conclusions by S, and by the proponents of his work? Anon, is this all the fault of "you people" who are always "making mistakes" and thus a "crowd" sort of too stupid for S to do "a favor [and] waste his own precious research time"? If so, why doesn't he just stop?

[Quoth]

Puffery and technically correct though misleading advertisements, though permitted in much commercial advertising, is not allowed for lawyers under state bar rules. Which standard should be aspired to by the law professors who purport to teach ethics and the schools holding themselves out as gatekeepers to an honorable profession?

[anon]

"Which standard should be aspired to by the law professors who purport to teach ethics and the schools holding themselves out as gatekeepers to an honorable profession?" Certainly not any requirements imposed by state bar rules, because many professors are not licensed to practice.

[Derek Tokaz]

Since no one has commented on the Solomon article cited above, I will, because I think it's a great example of one of the things the law school cheerleaders are getting fundamentally wrong.

"The foundation found that as of 2012, lawyers had high levels of job satisfaction and employment as well as high salaries."

That is the only mention of job satisfaction in the article. The rest is all unemployment and salaries. Nevermind the studies that paint a much less rosy picture of lawyer satisfaction (he hasn't even told us what "high" means). What makes Solomon's article uncompelling is that he talks about law not as a career or a calling, but as a job, a utilitarian means to pay the rent. We shouldn't be surprised that the 90s kids aren't lining up to spend $150,000 and 3 years of their life just to get an apartment with a second bathroom. Millennials have a different schedule of values, and making a lot of money tends to be pretty low.

And for the people who put income on the top of their list? An extra million over their lifetime isn't enough.

[[M][@][c][K]]

With respect to the Solomon article - I have commented on it elsewhere. I though it was an unfortunately tendentious article - and yes it cited Brian Leiter. Incidentally, if Simkovic wants to be considered as credibly and even handed, neutral commentator, using Leiter's blog as a base from which to make his points is hardly the way to do so, given Leiter's vituperative and partisan approach to the debate.

So the first problem with Solomon is his comment on falling enrolments: "Prof. Brian Leiter of the University of Chicago Law School predicts a 'bottom.'” Well Leiter predicts many things - but there is no evidence to suggest that a bottom is being reached. Indeed, classic market bubbles usually "bottom" below the level of long-run demand or price, and there is no evidence that the output of law graduates is at or below the long-term equilibrium level that the market can absorb. I think there is a way to go before applications "bottom" on that basis.

"The top global law firms ranked in the annual AmLaw 100 survey experienced a 4.3 percent increase in revenue in 2013 and a 5.4 percent increase in profit. Bigger firms are hiring. Above the Law, a website for lawyers, recently reported a rising trend for lateral moves for lawyers in New York."

Here there are a number of points that can be made. First, possibly the only data less reliable than the old USNWR data on post-law school employment and salaries is the AmLaw survey of law firm revenue's and profitability. I do not know many senior attorneys who believe the numbers, which are heavily based on un-audited self-reporting. Think about it, what managing partner is going to say our profits and revenues are down to AmLaw? Have you consider the concept of the death-spiral? If you don't believe me here is a quote form an article in Legal 360 (I won't put in a link so as to not upset the spam filter:

"A report that many of the country's top law firms may have overstated their average partner profits in The American Lawyer's annual rankings came as no surprise to legal industry experts, who have long viewed the published numbers with skepticism.
More than half of the top 50 law firms overstated their per-partner profits, and more than 22 percent of them inflated the numbers by 20 percent or more, according to a Wall Street Journal article published Monday. The Journal cited an analysis by Citi Private Bank Law Firm Group, a Citigroup Inc. unit that frequently lends to law firms, at a confidential meeting with law firm managing partners in Armonk, N.Y., earlier this month.

While widespread inflation of reported profits may be surprising to some observers, legal consultants including Jerry Kowalski of Kowalski & Associates say that industry insiders have long known that AmLaw's rankings are often inflated by law firms or based on unscientific guesswork.

“There's a universal understanding that these AmLaw numbers are completely meaningless,” Kowalski said. “They are more works of fiction than anything else.”

The rankings are a moneymaker for AmLaw, and still an industry-wide standard despite accuracy concerns, consultants say. Per-partner profit figures are a useful tool for lateral recruitment, as well as providing bragging rights for firms, according to Kowalski.

The widespread assumption that firms may fudge their profit numbers actually creates a more or less level playing field for comparison, as long as the numbers are viewed with appropriate skepticism, according to Bruce MacEwen, founder of the legal publication Adam Smith Esq.

“It may be rotten apples to rotten apples, but at least it's apples to apples,” he said."

So, in effect managing partners when it comes to data report, or at least law school publicists have been emulating law school deans.

The other issues I have to say were admirably addresses elsewhere.

[Jojo]

S&M's commentary regarding employment is why people hate lawyers.

"Employment" and "unemployed" are both official terms and a colloquial terms. To say that only 2 percent of lawyers are unemployed is deceptive, as it requires a knowledge of labor economics but is presented to the masses in the colloquial sense. if S&M report that figure in an economics journal, no big deal. When Directional State University puts it in a marketing pamphlet, it is slimy. Despite all the b.s. everyone knows this to be true, which is why it's being defended.

You only become an unemployed lawyer by first being a lawyer employee and then losing your job as a lawyer employee. Self employed lawyer = not an unemployed lawyer. Newbie law grad = not an unemployed lawyer.

And the post over at Prawfs that basically states that the 21 year old rubes should do more research before enrolling is sickening. You wonder why people are not enrolling and the quality of students is declining? Look in the mirror.

I've said it before, and I'll say it again - if some industry other than law school recruitment did what the law schools have done post-2005, the law schools would open up public interest clinics to try to remedy the wrong. Bully to Deb Merritt, Brian T, Paul C, and Bernie Burk for at least trying to reign in the nonsense.

[Derek Tokaz]

Jojo,

I also found the post at Prawfs a bit troubling. The post is claiming that while additional information might be helpful, that does not necessarily mean that summary information is misleading.

However, before making that claim, they point out that in order to understand the summary information, you need only do a google search about BLS terminology.

These seem to be contradictory positions. If it's not inherently deceptive, why is additional research needed? If without the additional research a prospective student is likely to be misinformed, then that would seem to be the very definition of inherently deceptive.

[BoredJD]

Guys, you're way off base.

21-year olds need to learn that the world has no sympathy for people who are not stock characters in an Econ 101 textbook. Of course, this lesson is best learned by taking out hundreds of thousands of dollars in non-dischargeable student loan debt at high interest rates, giving it to the people who pay my salary, and then shutting up and not trying to convince anyone else not to make the same mistake (that would be "whining").

That a lot of people making the same mistake over and over again happens to be in my pecuniary interest has nothing to do with my opinions on this topic. Literally nothing at all.

[[M][@][c][K]]

The post over at Prawfs is pretty appalling. To summarise - sponsored by the Law School Admission Council and ACCESS (a law school lender) Simkovic prepared an article with the used car salesman come on for its initial title of the "Million Dollar Law Degree," a title tailor made for assorted shills like Solomon et. al have used to tout going to law school. Indeed, given the sponsorship, Simkovic cannot have been unaware that the paper would be used for this purpose (let alone the follow on paper "Go to law school TODAY!"

To then sniff as BDG does, citing Simkovic "that the definition of unemployment can be googled (probably by an 8th-grader, but he says "by a college graduate") pretty easily -- a step, I might add, that might reasonably be expected of someone who is relying on data to decide how to spend 3 years of their life" shows a level of shamelessness that would elude a time share salesman. S&M were in the business of providing fodder for law school marketing - it was certain to be presented to prospective students.

Indeed if one looks at the Wikipedia Page (there is one) "Law school in the United States," one finds that Simkovic and McIntyre's study is all over its pages - while someone with a Fordham law IP address deletes every caveat on the reliability of the S&M data - and adds cheap shots and distorted quotes against Tamahama. A quick question to Fordham - is the application data that bad? It also presents BL as a legal sage of the utmost reliability while the S&M study is presented as outright fact. The page makes Tamahama and Campos sound controversial, but S&M and BL as saints (BL it should be noted edits Wikipedia constantly, including his own page.)

How is this putative googling college student to deal with the data when the likes of the mystery Fordham person (probably the admissions dean), BL and presumably S&M are constantly distorting the information available at supposedly neutral sites like Wikipedia?

[Voodoo94]

[M][@][c][K],

The mystery Fordham person could be Michael Simkovic himself. He is in a visiting status there this year.

[anon]

Good catch, Voodoo94. That whole post-graduate employment section is really just disgraceful.

[Bernie Burk]

And now a word from your Postmaster:

Okay, everyone, time to take a deep breath. Almost everyone in these Comments posts anonymously, and some of you are taking advantage of your anonymity to accuse others of bad behavior or heap insults in general terms from behind the security of screen names. If you have something to say about the substance of others' arguments or the data on which they rely, by all means have at it, but please try to attack the position and not the person. For that matter, a personal attack from someone else does not justify your response in kind.

I'm going to have to start deleting some of the Comments that stray too far. I don't do this lightly. Please try harder to stick to the issues.

Bernie

[Jojo]

Bernie,

Look at that Wikipedia entry. You'll want to take a shower.

Law faculty, I know we're talking about your jobs here, but almost all of you are decent folks who got into academia for a good reason. Presumably you care about students and teaching in addition to doing research. Almost none of you cared for 'caveat emptor' in law school or in the few years you were practicing before teaching. It is kind of a refuge for scoundrels. And while law students are not widows or orphans, you all know that 0Ls are not the most sophisticated consumers. They also revere most of you and your institutions. Caveat Emptor? You're all better than that. Let's rip the band aid off, get the truth out there, and build a better profession for the future. Unless we do, you're in for a death by a thousand cuts over the next 10 years and there will be a lot of naive students hurt.

[Derek Tokaz]

Rather than guessing who may be who or who's updating what on Wikipedia, I think it is useful to look at the way certain claims are sourced and how citations are used.

I'll start with a lesson I give to my freshmen about using sources: Citing a conclusion someone else reached is almost never sufficient support. It's only good support if that person is recognized as an expert and is speaking to something that isn't very controversial. For instance, "Gordon Ramsay says the proper way to cook Beef Wellington is..." Most of us would feel very confident that Ramsay got it right, and that more or less settles the matter for us. However, when that level of expertise and lack of controversy are missing, you must include not only your source's conclusion but also his reasoning behind it. For instance, "John Q. Chef thinks restaurants should keep their menus limited because this allows chefs to have better quality control." That "because" is clutch. Without it we don't know if the reasoning behind the conclusion is strong, weak, or totally absent. We would have no reason to trust the conclusion.

Now we can apply this to Solomon's reference to Leiter. Solomon writes, "Early figures show that applications for this year may be stabilizing. Prof. Brian Leiter of the University of Chicago Law School predicts a “bottom.” Applications are down only 2.9 percent so far this year from last year, according to the Law School Admissions Council, and it seems as if the law school entry class is likely to stabilize around 37,000 students — or back to 1970s levels."

Brian Leiter isn't an expert (that I know of) on law school enrollment trends, at least not so much that his prediction of the market hitting a bottom is worth listening to unless it is backed up by good reasoning and evidence.

If we follow the link, we see basically the same thing Solomon has said, that applications are down "only" 2.9 percent. What is the evidence to support the claim that they're going to stabilize? ...None is offered. An unsubstantiated claim doesn't suddenly gain credibility because it has cited someone else making the same unsubstantiated claim.

[anon]

On the first of these threads, on April 19, 2015 at 01:52 PM, Anon stated:

“DT: From the paper you refuse to read or are incapable of understanding - Table 3 reports several regressions that substitute alternate dependent variables for log earnings. We compute the hourly wage as total annual earnings divided by total hours worked and then use the log of this as our dependent variable. … In other words hours worked does not eliminate the (65%!) premium. Do you get it now?”

Perhaps I misread this comment.

[anon]

FE

Agreed. I missed the lack of quotation marks, and would have no objection to my comment being deleted. Mea culpa. ANd, I agree with your comment about not going down that road.

On the point of this thread, reading S's post on that other blog about how to define "unemployed" it is clear that he believes that any employment of a law graduate, full or part time, in a JD required job or not, should be counted as "employed" and further descriptions of such employment perhaps are better avoided:

"In many contexts, additional information might be useful to at least a minority of the audience, but too much additional information could be distracting, unnecessary, or even patronizing for the majority. There is a distinction between the potential for additional information to be useful and the stronger claim that summary information is inherently misleading."

Glass houses indeed.

[[M][@][c][K]]

Bernie:

Can I make a suggestion. For your next thread - how about a discussion of the Wikipedia entry "Law School in the United States" with particular reference to its very first section "Post-law school employment" and then "Controversies involving US law schools." One has to recognise that Wikipedia is a widely used reference by many people - but it does seem that determined efforts are being made to spin its contents.

Frankly, I would regard a solid review of the contents of the Wikipedia entry as a public service.

Inter alia, after you read those sections, as one deleted comment put it, you will want to take a shower.

[WikiThought]

We do all know that Wikipedia pages have an accompanying "talk" page where the contents of the page can be discussed within the context of the Wikipedia rules and goals right?