I was going to comment on Profs. Harrison's & Mashburn's draft Citations, Justifications, and the Troubled State of Legal Scholarship: An Empirical Study, but Al Brophy beat me to it. They note that law professors will instinctively push back against their results. Count me among them - at least some of the normative conclusions.
Interestingly, I agree with most of their empirical findings as well as many of their suggestions for improvement for the system. For example, it's not shocking to me that citations are higher to articles in higher ranked journals. To me, this is just an instantiation of the Matthew Effect. I've heard many argue that the better law reviews get cited because the articles are better, but the findings here, that most of the citations are for "facts" and not "engagement," belies that argument. (Note, as Harrison & Mashburn note, one can quibble with some of their classifications.). The fact that judicial citations are more distributed by rank also implies that the best articles (or at least most useful) are not necessarily in the higher ranked journals.
But where I diverge with the authors is what this all means. They take it to mean that citations are a bad way to assess scholarly value. I'm good with that. They also take it to mean that the scholarly enterprise is probably not worth the investment. (We'll leave aside their calcuation of the investment; I think they overestimate the marginal cost). I'm not sure that their results lead to that conclusion, for a couple of reasons.
First, I believe they understate the value of articles read but not cited. There are many uses of articles by professors and lawyers that never wind up being cited anywhere, though this is surely subject matter based. As the draft notes, SSRN downloads (and abstract views) far outstrip citations. These, too, are based on prestige, but that doesn't mean the articles being read have no value. Some ways there might be influence:
1. People might be influenced by ideas and just not cite them. Given the citation patterns found in the study, this is not out of the question.
2. Professors are teaching students, our future lawyers. Academic discourse and scholarship will affect what and how we teach subjects.
3. Scholarship might be used in legislative reform debates but never get cited.
4. Practitioners may read it and make arguments.
Of course, none of these will apply to every article, but they are value not measurable by citations. Indeed, it is odd that the article first argues that citations are not a good way to measure the value of scholarship and then uses lack of citations to argue that scholarship is valueless.
Second, and really the crux of my normative differences, I don't know that the authors have exhausted all the reasons for legal scholarship that they reject as unpersuasive. The article starts with a prior belief that scholarship has to be useful for something in order to be worth the while. Indeed, the notion that scholarship might be of interest to only other scholars is viewed with derision. Maybe this just bucks against the current "law school is a trade school whose only value is to get jobs for students" view, but I see such a limited view of legal scholarship as exceptionalism. The reality is that citation rates vary widely by discipline. Engineering sciences are right in the middle; maybe we should tell the engineers they should stop seeking all those government grants because no one ever cites their work. I've read hundreds (thousands?) of computer science articles. Many fascinating, many containing novel applications. Most did not disclose something that became commercial software. Perhaps they should just stop trying. And don't even ask about the humanities, with the worst citation rates of them all (well worse than law, I think). It seems that either you buy into the view that scholarship is worth doing of its own accord, or you don't. I'm sure there are plenty of people who would happily jettison humanities scholarship.
In the end, this article is convincing proof of the failure of its normative position. It is a well developed and executed empirical study that tells us something about the world that we didn't know. It might even be cited for those facts by someone talking about legal scholarship. But its normative angle likely will never be cited in a legal opinion. And, yet, it was worth the effort anyway.