Search the Lounge

Categories

« The Basis of the John Oliver Amicus Brief | Main | AALS CFP: New Voices In Legislation »

August 06, 2017

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Peer Review

I have published in both venues and there are advantages as well as disadvantages to both systems. I will address one point Frye makes which is his claim about submissions to law reviews and how they select papers. Peer review journals have serious flaws in the process the biggest one is the politics of the editor who initially screens your paper and decides, on his/her own, whether your work is deserving of sending out. While my evidence is personal and of course I may be wrong, U have had papers rejected within the hour if submission by peer journal EIC's who I am fairly certain exercised their discretion in a biased fashion. Why do I believe this? 1) the bios of the EIC and their extracurricular activities established their views and 2) the papers were published by T14 journals and have been well-cited.

Steve Lubet

There is a movement in the social sciences toward open archives and open peer review, including the SocArXiv in sociology: https://socopen.org/

One advantage of the peer review system is that its relative centralization makes it easier to reform. The most prestigious journals in sociology, for example, are published by the American Sociological Association, and specialty journals are published by ASA sections. Thus, reform, at least in theory, could be initiated top down (or by an insurgency).

That is more difficult, and maybe impossible, with law reviews, which operate independently and under constantly changing leadership. Even beneficial reforms that seem obvious and easy have been unable to endure. Take, for example, the announced preference for shorter articles, which most law reviews do not have the discipline to enforce.

Brian Frye

Steve,

Thanks for your comment. I have no illusions about the likelihood of actual reform. But it still seems like a conversation worth having. In theory, it would be relatively easy to make law review fairer and more pedagogically effective. I think you are right to predict that law reviews (editors & institutions both) would resist change. Unfortunately, I suspect many law professors would resist change as well. After all, most of them are former law review editors, including myself! But I think that the annual turnover of law review staff at least theoretically create a situation where new institutional norms could be established relatively quickly. As you suggest, it would probably require a centralized, top-down push. And I see no evidence that the relevant central "authorities" (AALS, I guess?) see this as a priority, or even something worth considering, so...

Doug Richmon

With respect to the inefficiency of law reviews, I am not sure that print publication is a material cause of publication delay. I recently accepted a publication offer from a fine law review that promised an April publication date and in late July the editors had yet to send print-ready articles to the printer. I have had many more similar experiences. For whatever reason, the editorial process too often bogs down horribly, whether because of students' busy schedules or uncooperative authors, but neither is a print medium problem.

Brian Frye

Doug,

Thanks for your comment. While I think that print publication is a needless & inefficient expense, I agree that print qua print is not a significant cause of delay. However, the perceived need to adhere to the traditional practice of releasing law reviews in print "issues" does introduce delays, at least in my experience, as delays in finishing one article apply to all the rest, as well as subsequent issues. With the exception of symposium "issues," I see no reason that law reviews shouldn't "publish" articles as soon as the editing process is complete. There is no longer any need to package them up in printed issues, which nobody wants or needs.

BLF

Daniel S. Goldberg

This is a terrific post. As someone who has ample experience publishing in both law reviews and peer reviewed journals, I think the section labeled "Incoherent" is right on the money in particular. Judging quality is difficult under the best of circumstances, but the structure of the law review system makes such assessments enormously more difficult. Moreover, because it is unmasked, it makes problematic cognitive biases much more likely (resulting in all the structural inequalities tracking what I call the 'usual social suspects').

(As I indicated in a prior comment on TFL, peer-reviewed systems have enormous problems of their own. But such a system is the worst one imaginable save all the other ones that have ever been tried ... Other systems, like that for law reviews, have more significant deficiencies and less benefits IMO)

Matthew Reid Krell

Brian,

How does your system allow hiring committees to discern whether a job candidate has the capacity to produce scholarship? (I submit that it does not, and thus would simply further harden the credentialism of hiring in legal academia that currently exists. Doesn't mean your system isn't an improvement over the current model, just means that here's a new problem that needs to be solved.)

Brian Frye

Matthew,

Thanks! That's a great point, which I totally failed to address. And it really should have occurred to me, given my reference to the discussion among philosophers. I guess I am farther from the job market than I realized.

A few possibilities jump to mind.

The most obvious is that non-faculty scholars could publish in the "law review" of the school from which they graduated. Perhaps "law reviews" would need to adopt a "third-category" of publications between "notes" and "articles"? My sense from publication pre-faculty & pre-tenure-track is that a de facto distinction already exists, so perhaps it would not be such a change? It could also provide an opportunity for post-graduation pedagogy for former students headed into academia. Some schools do this already & I think it would be good to encourage.

In the alternative, non-faculty scholars could "submit" to journals? I am inclined to think this is less than ideal, as I don't think the submission process makes much sense.

They could also "submit" to peer reviewed journals, but I don't think that alternative would work particularly well, either.

Or they could just write & post to SSRN or some other aggregator & "publish" after they get hired? Again less than ideal.

I would be very open to other suggestions on this front, which frankly hadn't occurred to me until you brought it up, and would definitely need to be addressed. Thanks again!

BLF

Brian Frye

Daniel,

Thanks! I'm glad you enjoyed it. I hope it revives the interesting conversation that Ann Bartow started. (As an aside, I had a twinge when I read all of the insightful comments Dan Markel made on the original posts.)

This post is my expression of a growing sense that we could do better. For ourselves, our students, and the audience (such as it is) for our scholarship. Personally, I am most concerned about my own students & the law review students I work with at other schools, who are incredibly enthusiastic & diligent, but also know that the experience could be better & more productive. I think we owe it to them to focus some attention on structural reform, in the interest of better pedagogy. And I am pretty confident that it could result in more inclusive, diverse & productive law review participation and scholarship.

BLF

Jeremy Telman

Agreed in part as to inefficient. Some of the delays are due to editing, and some of the student editing is actually valuable. I certainly have had the experience of having student editors catch erroneous citations or improve the clarity of my writing. When I was a student editor, I pushed authors to provide citations for claims that I could not verify. When they couldn’t we agreed to drop the meritless parts of otherwise meritorious scholarship.

There is no reason to publish hard copies of law reviews, but why stop with law reviews? Why do we still have casebooks? Why aren’t all treatises online only? Why aren’t they edited wikis so that interested academics and practitioners could update them as new cases are decided? Monographs are handy, but why are they not simultaneously published in electronic formats so as to be readily searchable?

As to ineffective, I find your argument extraordinary. As you note, “Lubet is right to observe that student law review editors catch errors and misrepresentations that peer reviewers miss. . .” You place this in a subordinate clause and then go on to complain about student editors’ footnote obsession. This reminds me of Adorno’s complaint that a German editor would never have complained that his essays were “poorly organized” as American editors did. Adorno misses the irony that a German editor (at the time) would have instead sent him to the camps. You object to editorial inquiries, but one can always tell a student to piss off. While you’re at it, why don’t you thank them for catching your mistakes, which a peer editor would never have discovered?

Whatever their history, here is the reality of student-edited law reviews today: Students who know little about the law use proxies and their own instincts to decide what to accept. Knowing nothing, they demand some authority for every claim. This is not a terrible thing, because judges and juries are also not experts, and so it is a good habit for a lawyer to know that what is obvious to her may not be obvious to the trier of fact.

As to “incoherent,” I think law reviews are, by design, guilty as charged. It would be a problem if a general review of any kind were anything but incoherent. I would have been disturbed if the American Historical Review published only articles about the U.S. South because its editorial board had a concentration of expertise in that area. As for the other problems, you seem to concede that law reviews are, because of letterhead bias and “suggestions,” at worst, as bad as peer review. But I don’t think that is the case across the board. Part of the ignorance of student editors is blissful. They don’t always know a big name when one crosses their path, but they know what they like, and that gives outsiders a chance to get in.

Certainly, a lot of “dross” (flashy or otherwise) gets published, but that being so it is hard to see how that would cause quality scholarship to be ignored. As you note, nobody reads law reviews. Mostly, we search electronic databases for articles in our field or we hear works in progress or meet colleagues at conferences and then follow their work. No amount of flashy dross is going to distract qualified scholars from finding the hidden gems. And so, I look on the multitude of publication opportunities as a boon to our field. A lot of dross gets published, but eventually everything gets published. That is not the case in other fields, where a couple negative reviews can doom a piece, as there are not that many relevant journals and peer review can drag on for six months or a year.

I think your suggestions for reform would be, at best, a sideways improvement. If I were tenured at Yale, I think I would be fine with a system that guaranteed my placement in the Yale Law Journal. If I were untenured at Yale, I think I would be frustrated with having only one venue in which to publish. But if I am not at Yale, limiting myself to my law school’s journal is distinctly unappealing, if I have been publishing or hope to publish at more highly regarded publications.

Finally, my main reason for defending student-edited law reviews. Law reviews can be a vital component of students’ educational experience.  Some of the best training that happens at law schools happens at law reviews.  I came to law school with ten years of scholarly experience under my belt, because I had written a doctoral dissertation, published historical scholarship and taught before making the jump to law school.  Still, my skills as a researcher skyrocketed in my third year as a law student when I was responsible for overseeing a team of cite and substance editors on a number of review essays that we published in our journal.  The evidentiary standards for legal scholarship are far more exacting than they are in the humanities and the non-quantitative social sciences.  No claim can be made without authority.  As a result, I became a far more intrepid researcher, and I unlearned intellectual habits acceptable to my former field of study and adopted intellectual habits essential to successful lawyering. 

Alexander Tsesis

I can't possibly say it any better than Jeremy Telman did, so I won't even try. But I want to add one anecdotal and hopefully motivational minutia:

Law reviews are a tremendous joy to read. I subscribe to about 75 journals through our library, and I find something interesting in about 3 to 5 out of every 10 journals I receive. Sometimes I only skim them, but quite often I read pieces all the way through. At other times, I solely read abstracts. This keeps me up to date in my field and aware of matters I'd never have otherwise read. Furthermore, often I run across symposia that have multiple articles on a specific topic of interest to me; whereas, if I had found one piece of a symposium online, I wouldn't have recognized all the other good and fascinating pieces in that volume (and of weak and sloppily drafted ones as well, but there's a value of reading those too).

So, I am a huge fan of printed volumes. I read them at home, on the train, and as I walk down the street. They save me from sitting at the computer all day and allow my eyes to rest a bit from the screen-light.

The motivational part of my post is to urge as many of you as have time to also get print copy through your libraries. To add one other reason for my preference, the final draft is always the cleanest one, while the drafts on SSRN are not always as clear of mistakes as the final versions. And for the additional reason that Steve and Jeremy mention concerning the tremendous benefit we get from student editor citation checks, I prefer print. That's not to say that I avoid PDF but that there's plenty of reason to call on those of you who don't currently have print subscriptions to give them a try. I'm willing to bet that the Academy would be a happier place for it and we'd become even more connected in our intellectual pursuits.

None of this is to say that there aren't significant problems with the law review process (letterhead bias, favoritism, etc.) but that the law review system is a worthy one that needs improvement not entire abandonment.

As for everything else, I refer back to Jeremy's post.

Brian Frye

Jeremy,

Thanks for your thoughtful (& lengthy!) comment. I think we largely agree on the merits, but some of my observations were poorly stated.

I could not agree more that most legal scholarship should not be published in hard copies. I only assign my students free or pay-what-you-wish online casebooks, or prepare my own materials. If they want to print them out, they are welcome to do so.

I also agree with your observations on student editors. I was trying to build on Lubet's observation that they are often (usually?) more effective at catching actual errors than peer reviewers. Primarily because they (more or less) are fact-checking, in the traditional sense. I think this is an incredibly valuable service! And I certainly didn't mean to criticize it. My point was merely that the editorial process could be better for students & scholars both. As it stands, scholars (typically professors) publish articles in journals edited by students from other schools. As a consequence, they never meet the students, do not have repeat interactions with the students, and have little or no pedagogical impact on the students. Which in my experience means a lot of wasted time for everyone. I would love to be able to work with my own students through the editorial process, to show them what is productive & why & how to use their time most effectively.

As a law review editor, I also had the experience of catching errors. And as an author I have occasionally (& embarrassingly!) made errors myself. But far more frequently, I have experienced students doing a lot of unnecessary work making edits & me doing a lot of unnecessary work undoing them. Some claims need substantiating citations, but some statements of fact do not. For example, what is purpose of demanding a citation for a footnote stating the years Gladstone was prime minister of England? (A real experience, btw.) Or a long string of "id" citations? If they were my own students, I could have a conversation with them about the purpose of citation & how to make it efficient, effective & useful.

I agree that the strength of law reviews is that (almost?) everything gets published. That is why I prefer them to peer review! My point is only that it could be even better. The process could be cleaner, faster & more equitable. I'd like to keep all of the things that are good about law review, but focus on minimizing the things that don't work.

In particular, I am concerned about the "prestige" issue. Ultimately, the point of the proposal I am making (as Ann Bartow recognized way back when!) is to eliminate "prestige" associations with law review "placement" by making it mechanical. Or perhaps, to eliminate the prestige element by making "publication" a pedagogical process between professors and their own students, rather than jockeying for position. Maybe it wouldn't work? That would certainly be a problem. But really my proposal is to eliminate "law reviews" as "brands" & reconstitute them as processes. In other words, scholars would really just publish "research papers" & the market for scholarship could determine what is valuable, without law review "prestige" intervening.

And I agree that law review is an important and beneficial part of the student experience. I just think it could be even better. And I think the easiest way to make it better is to make it an opportunity for students to interact with and learn from their own professors.

BLF

Brian Frye

Alexander,

Thanks for your comment. I also like to read articles in print. But I think it would be more efficient to focus on making sure that posted PDFs are accurate & let people print them out themselves.

I am also a big fan of symposia, not only because they collect articles on similar subjects, but also because they provide an opportunity for legal scholars to discuss specific topics with each other in a formal setting.

Perhaps the future of law reviews could be specialty journals focused on symposia? I would find that an agreeable possibility.

BLF

Jeremy Telman

Dear Brian,

We do seem to be agreed on just about everything except the conclusion. I think the problems with student editing that you identify are relatively trivial. I don’t like the extra ids. or having to footnote Gladstone’s dates, but such edits do very little damage to my work. My solution, if I find student edits irksome, is to reject them. I don’t get much pushback when I do. I tell student editors that theirs is a one-year gig, while my scholarship is with me for life.

No doubt the world of student-run journals could be improved, but it seems to me that what you want to achieve could be accomplished through better faculty advising of law reviews and the promotion of written guidelines about editing that would create institutional memories at journals. I think it is problematic to have faculty members overseeing students who are editing the faculty members’ work. There is a salubrious adversary element to the editing process which would be compromised by the power relationship between faculty members and their own students.

We have struck a balance in the field of legal scholarship. Faculty resources are expensive, so we have students work for free (or for credit). What we lose in expertise, we make up in volume. Any system that would increase faculty involvement would shift the balance, and I see no need to do so given that: 1) we have plenty of peer-reviewed journals in which we can publish; 2) law reviews have developed their own idiosyncratic mechanisms of faculty oversight; 3) one can always use one’s research assistants as student editors avant la lettre and achieve the pedagogical advantages you seek; and 4) I have some guarded faith in the wisdom of crowds – even non-experts, acting in concert, can separate the jewels from the dross.

Brian Frye

Jeremy,

Good points all. Perhaps my proposal is unnecessarily drastic. And it is certainly possibly that a student-teacher relationship would make it difficult for law reviews to produce the vetting benefits that you & Steve Lubet identify. I recently learned that Dave Hoffman has made some suggestions germane to your points, see:

https://concurringopinions.com/archives/2006/04/making_an_impac.html

and

https://concurringopinions.com/archives/2013/03/what-can-law-review-editors-do-to-attract-better-articles.html

The one troubling issue I think more modest reforms may have difficulty addressing is the "hierarchy" problem. Maybe we are stuck with it! But I fear it effectively blunts the ability of quasi-market processes to highlight good scholarship and scholars.

BLF

The comments to this entry are closed.

Bloggers Emereti

StatCounter

  • StatCounter
Blog powered by Typepad