Jeff Harrison and Amy Mashburn of the University of Florida have a paper up on ssrn, "Citations, Justifications, and the Troubled State of Legal Scholarship: An Empirical Study." It asks the uses that authors and courts make of legal scholarship. The ask whether scholarship is worth the cost, which they estimate at $240 million per year. Cribbing now from their abstract:
Recent pedagogical, economic and technological changes require law schools to reevaluate their resource allocations. Although typically viewed in terms of curricular changes, it is important also to focus on the very significant investment in legal scholarship and its impact. Typically this has been determined by some version of citation counting with little regard for what it means to be cited. This Article discusses why this is a deeply flawed measure of impact. Much of that discussion is based on an empirical study the authors conducted. The investigation found that citation by other authors is highly influenced by the rank of the review in which a work is published and the school from which the author graduated. Courts, on the other hand, are less sensitive to these markers of institutional authority. Perhaps more importantly, when the purpose of the citation is examined, a very small handful of those citing a work do so for anything related to the ideas, reasoning, methodology, or conclusions found in the cited work. This is slightly less true for judicial citation compared to citations by other authors. Given the level of current investment in legal scholarship and findings that reliance on it is far lower than citation counts would suggest, the authors offer a number of recommendation designed to increase accountability of legal scholars and the utility of what they produce.
I'm guessing there's going to be a lot of talk about this paper.
Since Harrison and Mashburn are saying that the costs of legal scholarship exceed the benefits, doesn't this line of reasoning apply to their own work too?
Posted by: Enrique Guerra Pujol | February 28, 2015 at 08:01 PM
"These assumptions result in an estimation of $960 million per year in law faculty salaries [including 30% in benefits]. One fourth of that figure--$240 million--can be attributed to the production of scholarship."
I'd be curious to hear if this is how other professors see the teaching/scholarship time allotment. What I've often heard is closer to a 40/50/10 divide between teaching/scholarship/admin duties (and where blogging during office hours fits into this is anyone's guess).
Given that a quarter of the year (give or take) is the summer, a professor could hit JH&AM's 25% mark with only minimal research throughout the school year -- only what is needed to offset lesson planning and grading done during the summer.
If a professor is spending a quarter of the regular school year on scholarship, as well as the vast majority (90%) of the summer, we get up to 41% of their time being scholarship, for $396 million per year. If it got up to a 50/50 divide in the school year, the professor would be spending 60% of his total time on scholarship, for $576 million. That's more than the National Cancer Institute spent on breast cancer research in 2013.
When it comes to measuring the value of that research, I agree with JH&AM that citations in academic works of are dubious value. Sometimes scholars really will offer valuable contributions to future scholarship, I don't think anyone doubts that this can and does happen. However, a mere citation is far from proof of that for any particular article. For instance, in response to the "Law Review Quote of the Day" thread a couple weeks back, I jokingly looked up some other silly quotes from the OP's own works. In one he writes "To be sure, the United States government [after 9/11] did not completely obliterate civil liberties," and cites an article by a UNC law professor. Not to diminish that professor's work, but I think we would all know that some civil liberties endured after 9/11 without any citation at all. Of the 100 academic citations the JH&AM study looked at (and I think this is too small a sample), 98 were junk.
Posted by: Derek Tokaz | March 02, 2015 at 09:07 AM
The legal academy seems to be obsessed with numbers, even though most lawyers have little to no expertise in quantitative analysis. It seems much more productive to look at the arguments and ideas shared and modified by legal scholars over time; just as in other fields, such scholarship advances the dialogue about important social, jurisprudential, and public policy issues incrementally, making it impossible to quantify the value of "an article" (or a number of articles, or a citation, or a lack of citations...)
Posted by: Valuation | March 02, 2015 at 11:29 AM
Valuation,
While I'm sure your approach sounds good to many, at the end of the day there is a decision to be made about how much money should be spent on legal research. There must be some criteria for that, and it ought to be something other than "however much we can get the kids to pay for."
And that reminds me of another point which goes into figuring out the cost of academic legal research: debt financing. That half a billion dollars is largely paid for with student loans, and as such the real cost is going to be something like 50% higher. It wouldn't be at all surprising to find that within a decade, academic legal research has run up a price tag of a billion dollars a year. In the meantime though, can we maybe spitball some ideas about how to get free labor from students in order to close the justice gap; or better yet, have them pay for the opportunity to work. I realize that's a bit off topic, not sure why it came to mind just now.
Posted by: Derek Tokaz | March 02, 2015 at 12:52 PM