LSAC Data and Predicting Number of Applicants for Fall 2015, Part 11

Amidst all of the excitement over U.S. News' new law school rankings (or maybe I should say amidst the discussion of whether people still care about U.S. News), I forgot to check on the latest on applicants from LSAC.  I now see that the LSAC is announcing that "As of 3/06/15, there are 269,840 fall 2015 applications submitted by 39,646 applicants. Applicants are down 4.6% and applications are down 6.5% from 2014.  Last year at this time, we had 76% of the preliminary final applicant count."  If this year's applicants follow last year's pattern, we will have approximately 52,166 applicants for fall 2015.  The last post in this series is here.

And in case you haven't yet had your fill of rankings talk for the week, here is a paper I put together that ranks law schools according to their median LSAT score, employment outcome, and law review citations.  I started out on it to see if I could replicate the U.S. News rankings using non-proprietary information and ended up thinking that I liked in particular the increased weight I gave to employment outcome.  I also thought that law review citations provided in some cases a better measure of a law school's intellectual environment that the notoriously static U.S. News peer assessment numbers.

Comments

[Anon]

Looks like things may slowly be stabalizing. Now, let's see if schools implement any real changes to tuition structure, more clarity and honesty in employment numbers, hiring faculty who are knowledge about and enjoy the practice of law, and the other reforms so many have discussed for so long, or if it will be business as usual.

[Derek Tokaz]

"I started out on it to see if I could replicate the U.S. News rankings using non-proprietary information..."

So what exactly is the ultimate goal of your ranking system? Who's the intended audience and how do you expect them to use it? (And assuming the answer is that you'd want prospective students to use as part of what they consider in making enrollment decisions, how does the cite count work towards that?)

[Al Brophy]

Thanks for asking, Derek. I originally thought of this as an experiment in replicating U.S. News. I was curious if there are publicly available data -- not the proprietary reputation surveys that U.S. News has -- that could be used as a replacement. And I was also curious about what a ranking "without the clutter" as Gordon Hylton phrases it, would look like. For me it was an experiment in what the data would reveal. Though I did not think of this initially as a question of audience, I'd say that I'm aiming this as the same people at U.S. News.

The law review citations? I see them as a way of gauging the school's intellectual community. I use them as a substitute for the U.S. News peer assessment scores, which seem too often to be based on long-out-date impressions of schools.

I blogged about this last summer. The initial iteration of the paper looked at only a subset of schools: http://www.thefacultylounge.org/2014/06/ranking-law-schools-based-on-lsat-employment-outcome-and-citations.html

Here's a post discussing the paper that includes all ABA accredited schools: http://www.thefacultylounge.org/2014/06/more-rankings-based-on-lsat-employment-and-citations.html

[John Jacob]

"Looks like things may slowly be stabalizing. "

I don't know that its a good thing that 2015 applications are stabilizing w.r.t. 2014 applications because of a February/March surge in applications. I'd imagine people who applied to law school after 2/1 aren't the brightest tools in the shed or aren't particularly committed to being lawyers. Wouldn't be surprised in an economic uptick sucked these folks back into the job market.

[JM]

"Looks like things may slowly be stabalizing."

Interesting quote. I would say that things have stabalized with the entry level job market for attorneys. I also agree that the decline in law school applicants is slowly stabalizing. So what is not stabalizing? The crushing burden of student debt, the decline in quality of law schools graduates, and the absolute vitriolic criticism of law schools by former grads and the media. I would expect all of these things to continue and get worse. Grads will fail the bar exam in record numbers. Elite grads will see tuition increases erode the value of high salaries. IBR/PAYE may be capped or repealed. In short, the future is still bleak if law schools don't enact some serious reform.

[Derek Tokaz]

Al,

While the audience of USN is up in the air at the moment as students seem to be using other tools to make enrollment decisions, I imagine the vast majority of its readers are still prospective students.

In that light, why would a prospective student care about the "intellectual community" at their school? Or to put it more concretely, why ought a prospective student care about the quality of articles being published in the school's primary law review?

Students who make LR might find it more rewarding to work on higher quality articles but that's a small benefit, and one enjoyed by few. Do they somehow benefit from professors holding their school in higher esteem? They would benefit from employers doing so, but we have employment stats which speak to that much better.

If the "intellectual community" is supposed to somehow speak to the quality of education, wouldn't it make more sense to look at the citations in the opposite direction? Not how many people are citing articles in your journal, but how many people are citing articles by your professors no matter where they are published?

[anon]

"But how many people are citing articles by your professors no matter where they are published?"

The Sisk (or Sisk-Leiter) study, updated semi-regularly, already does that:

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2109815

(Up to each person to decide whether they care; I'm just pointing out that there would be no point in Al doing this, since it's already done).

[Al Brophy]

Thanks for asking about this, Derek. I sure imagine prospective students care at least some about the intellectual community of a law school. Didn't you care about the community you were going to when you chose NYU? When I was looking at law schools now nearly 30 years ago I was interested in the courses that were available, the opportunities for extra-curricular activities, the quality of the other students and the faculty. Some of that, like the quality of students around you, is measured by LSAT score. Law review citations are another way of gauging what's going on at a school and how the school is perceived by others. Citations partly reflect the decisions of students in selecting articles (as well as the options the students had in what articles they could recruit) and they also reflect the scholarship that students write. Citations over the past seven or so years (I use the data John Doyle put together at W&L's law library's website) are more responsive to what's happening in the recent past at schools than are the U.S. News peer assessment scores. As I talk about in the rankings paper (and also in a paper on citations from a few year back http://papers.ssrn.com/sol3/papers.cfm?abstract_id=896313 ), there is a high correlation between citations to a school's flagship law review and its peer assessment score. One of the virtues of law review citations is that they provide a publicly available measure of a law school's peer reputation. There are other things one might add here -- such as citations to secondary journals, too. I tried a little along those lines in a short paper about a decade ago: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=959373

Are there other ways to measure intellectual community? Sure. Your suggestion of citations to faculty work is one of them and probably a pretty good one. In fact, I blogged some about the relationship between the citation study that Brian Leiter conducted (and that later Greg Sisk extended) and U.S. News peer assessment scores. Here's one of my posts: http://www.thefacultylounge.org/2010/09/scatterplot-of-siskleiter-scores-and-us-news-peer-assessment-1.html
And here's another one: http://www.thefacultylounge.org/2010/09/fun-with-scatterplots.html

That measure has a bias against newer faculty and thus has some of the same problems as the notoriously static peer assessment scores. Citations of faculty also doesn't reflect the contributions of students to the law review. More may be better, but I'm not sure that the addition of other variables -- particularly ones that correlate so highly with citations -- are necessary.

[Derek Tokaz]

Al,

I did what I imagine most prospective students do in considering schools (and I'll acknowledge that a lot of us will probably think that whatever we did is the norm). I looked at career trajectory and costs. I wanted to do transactional work in New York. The highest ranked school I got into was NYU (which was slightly higher at the time), and the next was UVA, and UVA offered me a little more money than NYU. But given the conventional wisdom of the time that law was recession proof and salaries were booming, I didn't worry about the extra debt, and went instead with the safer career play and picked NYU.

Intellectual community didn't factor in to it. Had I also been accepted by Columbia with a similar scholarship, then I probably would have looked more closely at the differences between the schools, but I wasn't. Very few people are deciding between Harvard and Yale, or Columbia and NYU. In most cases the three top concerns are location, job placement, and cost, and there are few close calls where intellectual community would serve as a tie breaker.

The intellectual community component is probably only really relevant to the elite of the elite who are planning on becoming professors. Then, knowing what the professors are doing at various schools is going to be hugely important.

But even if one were to really care about the academic community, how is the rank relevant to them? Surely anyone who cares and is making enrollment decisions doesn't just want a robust intellectual community. They care about the specific subject matter. 100 cites to civ pro articles is irrelevant to someone who wants to pursue criminal law. If you're scholarly interest is international tax, are you really doing to pick Harvard over NYU because it has a better intellectual community rank?

And going back to the topic of ideal metrics for a moment. Wouldn't it be better to look at the ratio of students to law review positions, secondary journal positions, moot court slots, clinic slots, and seminar seats? What good's a great intellectual community if you don't get to be part of it?

[Al Brophy]

Derek, lots to talk about here. On the big picture, which is ranking law schools (or should I say choosing between law schools), one of the things I think that you and Kyle are pushing -- or maybe Kyle more than you -- is getting a ranking of law schools that's adapted to each individual. This is sort of to law school selection what personalized medicine is to medicine. You're presenting information that helps people make a decision that's right for them. I think your point (going back to the discussion we had on correlation between employment outcomes and U.S. News rank) is that the U.S. News rank doesn't capture what's right for each person and maybe doesn't capture well employment outcomes for many schools. To that end, I'd urge people looking at any law school to look closely at it, taking in all sorts of aspects, not just placement, though certainly placement is incredibly important. People should should look at whether the law school they're choosing is the right one for them. You say that you didn't care about the intellectual community when you were choosing between UVA and NYU. Those two schools seem quite different to me on variables other than job placement. If those were the schools I was looking at, I'd weigh a bunch of things in addition to placement. But everyone's entitled to make choices on whatever variables they find most salient.

As I've said, law review citations are a proxy measure of community and a replacement for peer assessment scores (which are gauges of reputation). They correlated very highly with U.S. News' peer assessment score, so to the extent you're looking for a proxy only for U.S. News it's a good one. One could measure other things -- such as journal slots -- and maybe we should and maybe I will include that if I can get reliable data on that if I ever writing on law school rankings again. But that or number of moot court slots or clinic slots or seminar seats isn't the same thing as intellectual community or a school's reputation. What I'm trying to do is provide some alternative ways of thinking about the quality of law schools, which may be more responsive to the current realities that what U.S. News provides.

Thanks for participating in this discussion. I appreciate your engaging with this.

[Anon]

This is a great discussion. Thanks, especially since conversations about law school rankings and student choice often quickly devolve into nonsense and vitriol. The conversation highlights one point that possibly could benefit from further exploration. While those in the academy think that prospective students place substantial weight on intellectual community, that is really not the case. Instead, prospective law students, as reflected in the discussion above, primarily -- though not exclusively (think of future academics as an exception) -- care about career trajectory and costs. That is not surprising. Though law school should not be deemed a trade school, it should also not be treated as a future law professor proving ground; instead, law school is a professional graduate program, much like a MBA, that draws students who seek to use the program and the resulting degree as a means for professional betterment.

[Derek Tokaz]

Al,

Unless one is a future academic, I'd think giving intellectual community weight as more than a mere tie breaker would be monumentally boneheaded. You'd be improving the quality of your next three years, but risking the quality of the 40 years after that. I think it was Benjamin Franklin who said something about giving up essential security to gain temporary liberty, or something...

Imagine little Al Junior comes to you and says "Pops, I'm trying to decide between NYU and Alabama. Academically, I'm interested in international tax law. NYU is ranked 14 for citations, and what's more, it's known for its focus on international law and tax law. It's an academic home run! But professionally, I want to be a prosecutor in Alabama. Bama is ranked 47th for citations, and isn't known for its international tax program. On the other hand, I had a friend sneak a look at NYU's alumni database, and while there's 200 grads in Alabama listed, none of them are prosecutors. Heck, none of them are criminal defense attorneys either. It seems they're mostly tax types, and I don't have any interest in doing corporate work for my career. What school should I go to, pops?"

Even setting the cost issue aside, assuming NYU gives him a full ride plus a stipend, in what universe does it make sense for Al Junior to pick NYU over Alabama?

[Barry]

jm: "So what is not stabalizing? The crushing burden of student debt"

That seems to have stabilized at 'crushing and life destroying' for most grads.

[Barry]

Intellectual community doesn't matter if you can't get a position, or a position related to the field/location you want).

[Jojo]

It is just so bad out there for most lawyers. You really struggle to make a living. I understand why law faculty don't want to rock their stable boat, but I'd appreciate it if there were at least some (even anonymous) acknowledgement that it is Lord of the Flies out there for most practitioners.

[Adam]

Derek,

Your illustration is a straw man of the first order.

In general, I agree with many commentators in this forum that the legal academy has overweighted the attention it devotes to one of its constituencies - consumers of legal scholarship. That group is but one of many, and a recalibration, given the present darkness, is long overdue. So, I ought to be agreeing with the thrust of your comments here.

But your illustration imagines a world where there's this Abstract Stuff Prospective Students Find Cool (Maybe) vs. the Practical Stuff They Already Know They Need. I have taught law students for nearly twenty years, and a very small fraction went on to do exactly what they thought they'd be doing when they walked into law school. There are many reasons for that, but the biggest reason is that they realized they didn't really want to do X, Y or Z. And they liked something else better.

Your example is incomprehensible to me. A student who walks into law school with a deep interest in international tax law, whose ambition is to be a prosecutor in Alabama? How large a population do you suppose this is? Certainly, students develop interests in tax - or estate planning, or criminal law, etc. - during their time in law school. But your hypothetical student would probably benefit from attending a school known for its criminal law scholars, for such schools tend, in the nature of things, to orient their curricula towards subjects prominent faculty are interested in. I suppose I could imagine someone like your hypothetical who had a real interest in legal philosophy (which isn't exactly a practice area); but typical prospectives do not have solid or realistic ideas about what they will be doing as lawyers. For them, having an engaged faculty that can excite them to possibilities yet unexplored is important.

The key terms here are "typical", and "important". I agree that the student with unusually well-defined, considered and realistic career plans might choose to weight this criterion less heavily. She might be missing out on paths untaken, but a school meeting your more practical measures might be a great fit. And I can agree that far more law schools than is wise have elevated scholarly engagement far beyond tie-breaker status, in terms of resource allocations and costs. But your analysis treats intellectual community as utterly and entirely beside the point of law school. This is as persuasive as those faculty who would treat it as the only point of law school.

[[M][@][c][K]]

Adam:

In many respects I agree with you, except for the the implicit assumption in "they realized they didn't really want to do X, Y or Z. And they liked something else better." You assume a tremendous amount of agency on the part of young lawyers and law graduates.

The reality is that serendipity is a much larger factor in what law graduates end up doing - that and some level of suitability - than any conscious decision. You go to a law firm, you get a big assignment in antitrust - you learn some hard antitrust, they give you more antitrust, you suddenly are an antitrust lawyer. You get assigned to work on a merger, then another - you are an M&A lawyer, or telecoms, or whatever. The firm you manage to get hired by needs someone in white-collar, and lo and behold....

Someone I knew rather well wanted to be a litigator - great resumé, Fulbright Scholar, top grades from her law school, impressive Harvard undergrad - a little stiff and un-spontaneous, joined very very white shoe firm - they put her in corporate/securities - never moved her to litigation (privately she would have been a "good fit' by the firms standards, but not a great litigator at a senior level) - she stayed corporate. I see people who end up in telecoms, IP law, Dumping/CVD, labor law, etc. Relatively few wanted to do what they ended up doing - I mean few did not as such not want to do X or Y or Z, they were more ambivalent, but that is where the tasks that they had as junior lawyers, the assignment took them, they became known to be reliable, etc.

The reality is that legal employment out of law school is very catch-as-catch can. I became an expert in a field that I never studied in law school - it was what kept me employed for my first few years as a junior lawyer (and a good enough expert to be sought out by UNESCO and other international organisations.) Indeed a lot of the stuff I was regarded as an expert in I never heard of in law school, and I never went out to be an expert in - I just did it because we had a client who needed it and sometimes a partner who knew about it.

One of my critiques of some of the ideas about law school reform, of shrinking the curriculum, is that there are a lot of subjects I wish I had taken in law school, before I found myself doing them - but the courses were not available when I could do them, or the professors required us to take seminars in, well, their favorite non-subject. I think the law school curriculum needs to be as comprehensive as possible because it is so unpredictable where a legal career will take a law graduate. I worry that shrinking the curriculum will come at the cost of necessary subject matter - it is bad enough what law professors enthusiasms and desire to do anything but actual teaching of law has already done.

[Anons]

In response to the question above: every universe.

[Concerned_Citizen]

"Your example is incomprehensible to me. A student who walks into law school with a deep interest in international tax law, whose ambition is to be a prosecutor in Alabama? How large a population do you suppose this is?"

Hi Adam. I read that as being just one example.

I went to LS knowing (from a practical considerations standpoint) that I would practice IP and patent law, and so I have done for ~15 years. However, my intellectual interests in LS on the other hand tended more strongly toward family and child welfare law, and tax/WTE.

My two best study partners in LS were also patent attorney wannabes. Both are still doing that today. One was really jazzed intellectually by consumer protection laws (despite being a right-winger corporatist; go figure). The other professed no real interest outside of patents and TM.

So the idea of a student having on the one hand a particular professional interest and on the other a particular intellectual interest very different from the professional interest is not at all incomprehensible to me.

Of course, patent attorneys may be a special case, believing ab initio that they are largely already locked into a "best path" professionally.