The Past & Future of Law Reviews

I found Steve Lubet's recent post and essay reflecting on the relative merits of student-edited law reviews and peer-reviewed journals quite compelling, especially his observation that the law review editing process may help catch errors and misrepresentations that peer review can miss. And I can't help adding my own 2¢.

First, I want to echo Lubet's observation that peer review has many weaknesses, which make it a less than ideal choice. Historically, it probably increased the efficiency of the market for scholarship, by reducing information costs. But as information costs drop ever closer to zero, it becomes increasingly difficult to justify. For example, I was amused to see a recent post on the philosophy blog Daily Nous arguing that journals shouldn't publish articles by graduate students, because journals receive too many submissions and can't review or publish them all. Apparently, the author believes that preserving the peer review process is more important than the production and dissemination of ideas. If a process prevents the distribution of valuable ideas, surely the problem is with the process? While law reviews are inefficient, at least they are more efficient than peer reviewed journals!

Still, while I agree with Lubet that student-edited law reviews have under-appreciated merits, over the years, I've had many occasions to reflect on how they might be improved. As a law student, I was a member of the NYU Law Review and co-founded a new student-edited law review (NYU Journal of Law & Liberty). As a scholar, I co-authored an article presenting empirical research on law review copyright practices (Brian L. Frye, Christopher J. Ryan Jr. & Franklin L. Runge, An Empirical Study of Law Journal Copyright Practices, 16 J. Marshall Rev. Intell. Prop. L. 207 (2017)). As a teacher, I often work with law review members on their student notes. And I was recently approached by students who want to start a new law review. In each context, I have observed that customary law review practices are often inefficient, ineffective, and incoherent.

Inefficient

The inefficiency of the law review editorial process is legendary. While peer-reviewed journals may take even longer to publish articles, law reviews are still inexcusably slow. Many (most?) law professors post drafts of their papers to repositories like SSRN, Bepress, or the new Lawarxiv. Typically, articles do not appear in "print" until long after they are publicly available, often a year or more. By that time, most of the intended audience for the article has already seen and read (or ignored) it. Much of the delay is caused by the pointless convention that law review articles appear in printed "volumes" and "issues." Nobody wants a printed law review, especially a smorgasbord generalist one. It is a huge waste of time, money, and effort to produce print law reviews that inevitably go straight to the landfill, along with the law porn that accompanies them. There is no longer any reason for law reviews to publish anywhere other than online. If authors actually want printed copies of their articles, they can order them print on demand.

Ineffective

While Lubet is right to observe that student law review editors catch errors and misrepresentations that peer reviewers miss, they do so only because and to the extent that they are effectively fact-checkers. That is great, because most academic publishing lacks any fact-checking at all. But student law review editors aren't always particularly effective fact-checkers, especially because law review norms typically encourage them to apply a formula and check boxes, rather than think about what they are editing and ask what needs verification and what does not. Law review editors have internalized (institutionalized?) a ridiculous norm of demanding a citation for every statement in an article, no matter how trivial or banal. (I discuss this phenomenon in my article Plagiarism is Not a Crime, see fn. 127 for a laugh). Indeed, editors often even ask authors to provide citations for claims that anyone who had actually read (or understood?) the article would know was the author's (putative) contribution to the literature.

Incoherent

But the most fundamental problem with law reviews is the incoherent and seemingly arbitrary way in which they choose what they will publish. Unlike any other discipline, legal scholars have (ostensibly) delegated to law students the responsibility to evaluate the merit of their scholarship. In theory, it is charmingly humbling, even if it doesn't make any sense. There is something to be said for the spectacle of law professors angsting over whether law students will like their article, even if the students typically don't understand what they are reading or the context in which it was written.

And yet, everyone knows that it is really a charade. Letterhead bias and "suggestions" from faculty members ensure that students more or less get the message what to publish and what to ignore. Big names immediately get attention, and so do their proteges. That is par for the course, and no different than any other system, including peer review. But it is also unfair and potentially discriminatory. Many people believe (in my opinion with good reason) that it effectively - if not intentionally - enables and perpetuates discrimination against articles written by women and minorities.

The system is also remarkably noisy. A lot of good scholarship gets ignored, especially on subjects law students don't understand, and a lot of flashy dross gets published. It is an article of faith among law professors that law review editors prefer constitutional law to any other subject, and the odds of placing an article are proportional to the number of editors who have taken the relevant class. Law students also reward articles with lots of carefully bluebooked citations, a metric that seems largely uncorrelated with good scholarship. And under the wildly inefficient and depressing "expedite" tradition, most "prestige" law reviews don't even consider or bother reading articles until one of their "prestige competitors" has accepted it for publication.

So why is there a law review "submission" system at all? Ostensibly, quality assessment. But law students were never particularly good judges of the quality of legal scholarship, and it makes no sense to leave them as "gatekeepers" if superior alternatives are available. And are there ever! In practice, no one actually uses the "prestige" of a law review as a proxy for the likely usefulness of what it publishes. Indeed, no one "reads" law reviews at all. The only time law review "prestige" is used as a proxy for anything at all is in tenure decisions and the like, when the people leaning on it really ought to be reading the articles themselves, rather than rely on the judgment of law students. If anything, a "prestigious" placement is primarily evidence of ability and willingness to game a broken system and network with insiders. Probably useful skills, but also ones we might not want to reward even further.

The History of Law Reviews

Law reviews were created to provide a way of disseminating legal scholarship to lawyers. Initially, they were experimental and took different forms and approaches, different law reviews adopting different approaches, hoping to identify what their audiences needed and wanted.  Eventually, standard formats developed, although law reviews continued to evolve over time. In any case. the law review format developed in relation to print media and its traditions were intended to solve problems relevant to print media and its distribution.

Among other things, people needed to know what was worth paying for and reading and what wasn't. Law reviews provided a highly imperfect, but still useful rubric, given that there was little else available. If an article was in the Harvard Law Review, you could safely assume it was reasonably reliable and likely to be worth reading. And other law reviews also developed reputations for quality (or lack thereof), whether justified or not.

This basic model has persisted for more than a century. Until relatively recently, its mechanics were largely unchallenged, because there was no obviously superior alternative. But then we got the internet. And law reviews soon became institutional anachronisms, persisting not because they were efficient, but because they benefited certain insiders. Or just because. Which is all too often reason enough in legal academia, and academia in general.

Reform

In any case, I think law reviews could be more efficient and more effective. And I'd like to re-introduce an idea suggested by Ann Bartow in a Madisonian post way back in 2008: Why not eliminate the "submissions" process entirely, and just have law professors "publish" their articles in their "home" journal? (See also these Prawfsblog posts) (PS Thanks to Scott Boone for alerting me to Ann's post & an assortment of suggestions I received on Twitter). As Ann observed, doing so could eliminate many of the problems associated with student-edited law reviews by precluding letterhead bias and discrimination, and (eventually?) eliminating the "prestige" associated with publishing in journals associated with particular schools.

Indeed, with the benefit of hindsight, I think Ann's suggestion looks even better today than when she made it. I think one of the biggest benefits would be to increase the pedagogical value of law review to the students who do the work. Currently, they "edit" articles written by professors at different schools, who they will mostly never meet, and will interact with only once. If law reviews were to only publish articles written by professors from their own schools, the students would be able to sit down with the professors and talk to them directly about their articles and the editing process. This would be a great learning opportunity for the students. And it would encourage professors to mentor students more closely, and teach them about relevant scholarly norms. For example, professors could explain to students when citations are necessary and when they are not. Perhaps we could avoid the interminable parade of useless "id." citations and filler citations currently included in most law review articles.

Ironically, in many ways, this would represent a return to the law reviews of the 19th century, which mostly published articles written by their own law school professors and graduates. Back to the future! Everything old is new again.

Of course, under this model law reviews would be irrelevant. Why have general "law reviews" at all, when some open access repository would do the same thing just as well? I think that is a good thing. Law review editors could be more like research assistants for their professors, which would surely benefit both the students and the professors. And "law review" activities could be focused on symposia and so on, which offer far more potential for pedagogical benefit than the current general law review process. We should have more "specialty" journals focused on promoting productive conversations, and fewer (no?) "general" journals, focused on perpetuating hierarchies.

Yes, the transition would be difficult. To the extent that prestige bias persists, it could harm professors at less prestigious schools. And the small number of independent law reviews, like the Harvard Law Review would surely resist the change. Oh, who am I kidding, every law review would resist the change; there is nothing that law review editors hate more than change, no matter the reason for it.

But so what? If a system is broken, inefficient, discriminatory, and ineffective, I think we should change it. And we should explain to students why we are changing it. I think we can bring them around. But what about ourselves?

Comments

[Peer Review]

I have published in both venues and there are advantages as well as disadvantages to both systems. I will address one point Frye makes which is his claim about submissions to law reviews and how they select papers. Peer review journals have serious flaws in the process the biggest one is the politics of the editor who initially screens your paper and decides, on his/her own, whether your work is deserving of sending out. While my evidence is personal and of course I may be wrong, U have had papers rejected within the hour if submission by peer journal EIC's who I am fairly certain exercised their discretion in a biased fashion. Why do I believe this? 1) the bios of the EIC and their extracurricular activities established their views and 2) the papers were published by T14 journals and have been well-cited.

[Steve Lubet]

There is a movement in the social sciences toward open archives and open peer review, including the SocArXiv in sociology: https://socopen.org/

One advantage of the peer review system is that its relative centralization makes it easier to reform. The most prestigious journals in sociology, for example, are published by the American Sociological Association, and specialty journals are published by ASA sections. Thus, reform, at least in theory, could be initiated top down (or by an insurgency).

That is more difficult, and maybe impossible, with law reviews, which operate independently and under constantly changing leadership. Even beneficial reforms that seem obvious and easy have been unable to endure. Take, for example, the announced preference for shorter articles, which most law reviews do not have the discipline to enforce.

[Brian Frye]

Steve,

Thanks for your comment. I have no illusions about the likelihood of actual reform. But it still seems like a conversation worth having. In theory, it would be relatively easy to make law review fairer and more pedagogically effective. I think you are right to predict that law reviews (editors & institutions both) would resist change. Unfortunately, I suspect many law professors would resist change as well. After all, most of them are former law review editors, including myself! But I think that the annual turnover of law review staff at least theoretically create a situation where new institutional norms could be established relatively quickly. As you suggest, it would probably require a centralized, top-down push. And I see no evidence that the relevant central "authorities" (AALS, I guess?) see this as a priority, or even something worth considering, so...

[Doug Richmon]

With respect to the inefficiency of law reviews, I am not sure that print publication is a material cause of publication delay. I recently accepted a publication offer from a fine law review that promised an April publication date and in late July the editors had yet to send print-ready articles to the printer. I have had many more similar experiences. For whatever reason, the editorial process too often bogs down horribly, whether because of students' busy schedules or uncooperative authors, but neither is a print medium problem.

[Brian Frye]

Doug,

Thanks for your comment. While I think that print publication is a needless & inefficient expense, I agree that print qua print is not a significant cause of delay. However, the perceived need to adhere to the traditional practice of releasing law reviews in print "issues" does introduce delays, at least in my experience, as delays in finishing one article apply to all the rest, as well as subsequent issues. With the exception of symposium "issues," I see no reason that law reviews shouldn't "publish" articles as soon as the editing process is complete. There is no longer any need to package them up in printed issues, which nobody wants or needs.

BLF

[Daniel S. Goldberg]

This is a terrific post. As someone who has ample experience publishing in both law reviews and peer reviewed journals, I think the section labeled "Incoherent" is right on the money in particular. Judging quality is difficult under the best of circumstances, but the structure of the law review system makes such assessments enormously more difficult. Moreover, because it is unmasked, it makes problematic cognitive biases much more likely (resulting in all the structural inequalities tracking what I call the 'usual social suspects').

(As I indicated in a prior comment on TFL, peer-reviewed systems have enormous problems of their own. But such a system is the worst one imaginable save all the other ones that have ever been tried ... Other systems, like that for law reviews, have more significant deficiencies and less benefits IMO)

[Matthew Reid Krell]

Brian,

How does your system allow hiring committees to discern whether a job candidate has the capacity to produce scholarship? (I submit that it does not, and thus would simply further harden the credentialism of hiring in legal academia that currently exists. Doesn't mean your system isn't an improvement over the current model, just means that here's a new problem that needs to be solved.)

[Brian Frye]

Matthew,

Thanks! That's a great point, which I totally failed to address. And it really should have occurred to me, given my reference to the discussion among philosophers. I guess I am farther from the job market than I realized.

A few possibilities jump to mind.

The most obvious is that non-faculty scholars could publish in the "law review" of the school from which they graduated. Perhaps "law reviews" would need to adopt a "third-category" of publications between "notes" and "articles"? My sense from publication pre-faculty & pre-tenure-track is that a de facto distinction already exists, so perhaps it would not be such a change? It could also provide an opportunity for post-graduation pedagogy for former students headed into academia. Some schools do this already & I think it would be good to encourage.

In the alternative, non-faculty scholars could "submit" to journals? I am inclined to think this is less than ideal, as I don't think the submission process makes much sense.

They could also "submit" to peer reviewed journals, but I don't think that alternative would work particularly well, either.

Or they could just write & post to SSRN or some other aggregator & "publish" after they get hired? Again less than ideal.

I would be very open to other suggestions on this front, which frankly hadn't occurred to me until you brought it up, and would definitely need to be addressed. Thanks again!

BLF

[Brian Frye]

Daniel,

Thanks! I'm glad you enjoyed it. I hope it revives the interesting conversation that Ann Bartow started. (As an aside, I had a twinge when I read all of the insightful comments Dan Markel made on the original posts.)

This post is my expression of a growing sense that we could do better. For ourselves, our students, and the audience (such as it is) for our scholarship. Personally, I am most concerned about my own students & the law review students I work with at other schools, who are incredibly enthusiastic & diligent, but also know that the experience could be better & more productive. I think we owe it to them to focus some attention on structural reform, in the interest of better pedagogy. And I am pretty confident that it could result in more inclusive, diverse & productive law review participation and scholarship.

BLF

[Jeremy Telman]

Agreed in part as to inefficient. Some of the delays are due to editing, and some of the student editing is actually valuable. I certainly have had the experience of having student editors catch erroneous citations or improve the clarity of my writing. When I was a student editor, I pushed authors to provide citations for claims that I could not verify. When they couldn’t we agreed to drop the meritless parts of otherwise meritorious scholarship.

There is no reason to publish hard copies of law reviews, but why stop with law reviews? Why do we still have casebooks? Why aren’t all treatises online only? Why aren’t they edited wikis so that interested academics and practitioners could update them as new cases are decided? Monographs are handy, but why are they not simultaneously published in electronic formats so as to be readily searchable?

As to ineffective, I find your argument extraordinary. As you note, “Lubet is right to observe that student law review editors catch errors and misrepresentations that peer reviewers miss. . .” You place this in a subordinate clause and then go on to complain about student editors’ footnote obsession. This reminds me of Adorno’s complaint that a German editor would never have complained that his essays were “poorly organized” as American editors did. Adorno misses the irony that a German editor (at the time) would have instead sent him to the camps. You object to editorial inquiries, but one can always tell a student to piss off. While you’re at it, why don’t you thank them for catching your mistakes, which a peer editor would never have discovered?

Whatever their history, here is the reality of student-edited law reviews today: Students who know little about the law use proxies and their own instincts to decide what to accept. Knowing nothing, they demand some authority for every claim. This is not a terrible thing, because judges and juries are also not experts, and so it is a good habit for a lawyer to know that what is obvious to her may not be obvious to the trier of fact.

As to “incoherent,” I think law reviews are, by design, guilty as charged. It would be a problem if a general review of any kind were anything but incoherent. I would have been disturbed if the American Historical Review published only articles about the U.S. South because its editorial board had a concentration of expertise in that area. As for the other problems, you seem to concede that law reviews are, because of letterhead bias and “suggestions,” at worst, as bad as peer review. But I don’t think that is the case across the board. Part of the ignorance of student editors is blissful. They don’t always know a big name when one crosses their path, but they know what they like, and that gives outsiders a chance to get in.

Certainly, a lot of “dross” (flashy or otherwise) gets published, but that being so it is hard to see how that would cause quality scholarship to be ignored. As you note, nobody reads law reviews. Mostly, we search electronic databases for articles in our field or we hear works in progress or meet colleagues at conferences and then follow their work. No amount of flashy dross is going to distract qualified scholars from finding the hidden gems. And so, I look on the multitude of publication opportunities as a boon to our field. A lot of dross gets published, but eventually everything gets published. That is not the case in other fields, where a couple negative reviews can doom a piece, as there are not that many relevant journals and peer review can drag on for six months or a year.

I think your suggestions for reform would be, at best, a sideways improvement. If I were tenured at Yale, I think I would be fine with a system that guaranteed my placement in the Yale Law Journal. If I were untenured at Yale, I think I would be frustrated with having only one venue in which to publish. But if I am not at Yale, limiting myself to my law school’s journal is distinctly unappealing, if I have been publishing or hope to publish at more highly regarded publications.

Finally, my main reason for defending student-edited law reviews. Law reviews can be a vital component of students’ educational experience.  Some of the best training that happens at law schools happens at law reviews.  I came to law school with ten years of scholarly experience under my belt, because I had written a doctoral dissertation, published historical scholarship and taught before making the jump to law school.  Still, my skills as a researcher skyrocketed in my third year as a law student when I was responsible for overseeing a team of cite and substance editors on a number of review essays that we published in our journal.  The evidentiary standards for legal scholarship are far more exacting than they are in the humanities and the non-quantitative social sciences.  No claim can be made without authority.  As a result, I became a far more intrepid researcher, and I unlearned intellectual habits acceptable to my former field of study and adopted intellectual habits essential to successful lawyering. 

[Alexander Tsesis]

I can't possibly say it any better than Jeremy Telman did, so I won't even try. But I want to add one anecdotal and hopefully motivational minutia:

Law reviews are a tremendous joy to read. I subscribe to about 75 journals through our library, and I find something interesting in about 3 to 5 out of every 10 journals I receive. Sometimes I only skim them, but quite often I read pieces all the way through. At other times, I solely read abstracts. This keeps me up to date in my field and aware of matters I'd never have otherwise read. Furthermore, often I run across symposia that have multiple articles on a specific topic of interest to me; whereas, if I had found one piece of a symposium online, I wouldn't have recognized all the other good and fascinating pieces in that volume (and of weak and sloppily drafted ones as well, but there's a value of reading those too).

So, I am a huge fan of printed volumes. I read them at home, on the train, and as I walk down the street. They save me from sitting at the computer all day and allow my eyes to rest a bit from the screen-light.

The motivational part of my post is to urge as many of you as have time to also get print copy through your libraries. To add one other reason for my preference, the final draft is always the cleanest one, while the drafts on SSRN are not always as clear of mistakes as the final versions. And for the additional reason that Steve and Jeremy mention concerning the tremendous benefit we get from student editor citation checks, I prefer print. That's not to say that I avoid PDF but that there's plenty of reason to call on those of you who don't currently have print subscriptions to give them a try. I'm willing to bet that the Academy would be a happier place for it and we'd become even more connected in our intellectual pursuits.

None of this is to say that there aren't significant problems with the law review process (letterhead bias, favoritism, etc.) but that the law review system is a worthy one that needs improvement not entire abandonment.

As for everything else, I refer back to Jeremy's post.

[Brian Frye]

Jeremy,

Thanks for your thoughtful (& lengthy!) comment. I think we largely agree on the merits, but some of my observations were poorly stated.

I could not agree more that most legal scholarship should not be published in hard copies. I only assign my students free or pay-what-you-wish online casebooks, or prepare my own materials. If they want to print them out, they are welcome to do so.

I also agree with your observations on student editors. I was trying to build on Lubet's observation that they are often (usually?) more effective at catching actual errors than peer reviewers. Primarily because they (more or less) are fact-checking, in the traditional sense. I think this is an incredibly valuable service! And I certainly didn't mean to criticize it. My point was merely that the editorial process could be better for students & scholars both. As it stands, scholars (typically professors) publish articles in journals edited by students from other schools. As a consequence, they never meet the students, do not have repeat interactions with the students, and have little or no pedagogical impact on the students. Which in my experience means a lot of wasted time for everyone. I would love to be able to work with my own students through the editorial process, to show them what is productive & why & how to use their time most effectively.

As a law review editor, I also had the experience of catching errors. And as an author I have occasionally (& embarrassingly!) made errors myself. But far more frequently, I have experienced students doing a lot of unnecessary work making edits & me doing a lot of unnecessary work undoing them. Some claims need substantiating citations, but some statements of fact do not. For example, what is purpose of demanding a citation for a footnote stating the years Gladstone was prime minister of England? (A real experience, btw.) Or a long string of "id" citations? If they were my own students, I could have a conversation with them about the purpose of citation & how to make it efficient, effective & useful.

I agree that the strength of law reviews is that (almost?) everything gets published. That is why I prefer them to peer review! My point is only that it could be even better. The process could be cleaner, faster & more equitable. I'd like to keep all of the things that are good about law review, but focus on minimizing the things that don't work.

In particular, I am concerned about the "prestige" issue. Ultimately, the point of the proposal I am making (as Ann Bartow recognized way back when!) is to eliminate "prestige" associations with law review "placement" by making it mechanical. Or perhaps, to eliminate the prestige element by making "publication" a pedagogical process between professors and their own students, rather than jockeying for position. Maybe it wouldn't work? That would certainly be a problem. But really my proposal is to eliminate "law reviews" as "brands" & reconstitute them as processes. In other words, scholars would really just publish "research papers" & the market for scholarship could determine what is valuable, without law review "prestige" intervening.

And I agree that law review is an important and beneficial part of the student experience. I just think it could be even better. And I think the easiest way to make it better is to make it an opportunity for students to interact with and learn from their own professors.

BLF

[Brian Frye]

Alexander,

Thanks for your comment. I also like to read articles in print. But I think it would be more efficient to focus on making sure that posted PDFs are accurate & let people print them out themselves.

I am also a big fan of symposia, not only because they collect articles on similar subjects, but also because they provide an opportunity for legal scholars to discuss specific topics with each other in a formal setting.

Perhaps the future of law reviews could be specialty journals focused on symposia? I would find that an agreeable possibility.

BLF

[Jeremy Telman]

Dear Brian,

We do seem to be agreed on just about everything except the conclusion. I think the problems with student editing that you identify are relatively trivial. I don’t like the extra ids. or having to footnote Gladstone’s dates, but such edits do very little damage to my work. My solution, if I find student edits irksome, is to reject them. I don’t get much pushback when I do. I tell student editors that theirs is a one-year gig, while my scholarship is with me for life.

No doubt the world of student-run journals could be improved, but it seems to me that what you want to achieve could be accomplished through better faculty advising of law reviews and the promotion of written guidelines about editing that would create institutional memories at journals. I think it is problematic to have faculty members overseeing students who are editing the faculty members’ work. There is a salubrious adversary element to the editing process which would be compromised by the power relationship between faculty members and their own students.

We have struck a balance in the field of legal scholarship. Faculty resources are expensive, so we have students work for free (or for credit). What we lose in expertise, we make up in volume. Any system that would increase faculty involvement would shift the balance, and I see no need to do so given that: 1) we have plenty of peer-reviewed journals in which we can publish; 2) law reviews have developed their own idiosyncratic mechanisms of faculty oversight; 3) one can always use one’s research assistants as student editors avant la lettre and achieve the pedagogical advantages you seek; and 4) I have some guarded faith in the wisdom of crowds – even non-experts, acting in concert, can separate the jewels from the dross.

[Brian Frye]

Jeremy,

Good points all. Perhaps my proposal is unnecessarily drastic. And it is certainly possibly that a student-teacher relationship would make it difficult for law reviews to produce the vetting benefits that you & Steve Lubet identify. I recently learned that Dave Hoffman has made some suggestions germane to your points, see:

https://concurringopinions.com/archives/2006/04/making_an_impac.html

and

https://concurringopinions.com/archives/2013/03/what-can-law-review-editors-do-to-attract-better-articles.html

The one troubling issue I think more modest reforms may have difficulty addressing is the "hierarchy" problem. Maybe we are stuck with it! But I fear it effectively blunts the ability of quasi-market processes to highlight good scholarship and scholars.

BLF