Search the Lounge

Categories

« Arkansas - Fayetteville Law Seeks Entry Or Junior Lateral Faculty Candidates | Main | Ronald Coase (1910-2013) »

September 02, 2013

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Matt Lister

"this means that instead of a handful of elite gatekeepers determining what ideas other scholars can read, we can routinely get published some pieces that genuinely buck the conventional wisdom. (Note that I am not implying here some malevolent cabal or pernicious old boys' network; I just mean that the very people whose ideas have created the conventional wisdom are naturally unlikely to be impressed by outlier ideas that threaten it)."


I don't have a strong animosity towards student-edited law reviews. I'm a "let a thousand flowers bloom" sort of guy towards publication venues. (*) But I want to suggest that this is at least a somewhat misleading characterization of peer review, at least in the areas I work in.(**) I have reviewed articles for at least a half dozen journals in the last year- maybe twice that many. To call me an "elite gatekeeper" would be... more than a bit of a stretch. I also would like to think that I'm not someone who opposes pieces that "buck the conventional wisdom", though I suppose that's something people are less likely to be clear judges about when applied to themselves. I can at least say that I have recommended publishing papers that took views that were strongly opposed to my own, and recommended rejecting some that had views closer to my own when I thought the arguments were no good. I would not want to suggest that there is nothing to this characterization. Peer review can, in particular, be conservative. But I don't think that I'm a great outlier here, so think that, at least, the characterization paints in too broad of stokes.

(*) My own, not completely unbiased, impression is that law faculties tend to under-value peer reviewed publications, and that there is at least as much, if not more, bias in the opposite direction as suggested here.

(**) I'm usually, but not always, asked to review papers w/ a "philosophical" aspect, though these have been for a wide variety of journals, including straight philosophy journals, political theory/science journals, internationally based law journals (where peer review is much more common), and interdisciplinary journals of different sorts. I mention this just to suggest that the sort of practice I discuss is likely common among a wide range of fields.

Howard Wasserman

If Justin's model is right, then why not bypass law reviews and student editors altogether? Let everyone self-publish and distribute on a site such as SSRN, then see which ideas are accepted (and cited) post-publication?

anon

Howard:
Isn't there some thought that the student editors at "elite" schools (future Supreme Court clerks all) are as able to find and validate good scholarship as anyone else?
Given the number of journals, we now know that a piece will be published, somewhere, no matter the quality, which is somewhat like the self-publishing model you ask about.
That model now exists, in any event.
Better to self-publish on SSRN or in a 4th tier journal?

Howard Wasserman

4th tier journal, because that remains the norm. Change the norm (which Justin's post suggests would shift evaluation to the back-end) and maybe the answer changes.

Several years ago--in the relatively early days of SSRN and the even-earlier days of blogs--I recall some online discussion of a world in which each journal would publish the work of its own faculty.

Steven Lubet

Nearly all law review articles claim to buck the conventional wisdom, usually in the first paragraph, if not in the abstract. How able are students to evaluate such claims (given their relative unfamiliarity with the conventional wisdom in the first place)?

Steve Black

The lack of "peer review" is one issue; we also lack a blind selection process.

Anon

Steve Black has hit the nail on the head. The system lacks legitimacy unless and until the selection process can be viewed as impartial. If you dont believe there is bias in the system, you are deluding yourself.

Larry Rosenthal

The process of post-publication review that Professor Long describes is rather a hit-or-miss thing, or so it seems to me. It depends on the willingness of other scholars to take the time to scrutinize critically and with rigor the scholarship of others. I wonder how often this happens. William Stuntz was one of the most prolific and cited criminal procedure scholars in recent decades, but his work often contained broad empirical generalizations with questionable empirical support -- the kind of work that might be questioned by the editors of a peer-reviewed criminology journal, but which law review editors were eager to publish, and legal scholars were eager to cite. When Stephen Schulhofer undertook a systematic examination of Stuntz's empirical claims in his piece for the Michigan Law Review's Book Review issue, however, he dismantled a great many of Stuntz's central empirical claims. This is the kind of thing that rigorous peer review would be more likely to catch.

An example of this problem is found in this very post. Professor Long tells us that a world without student-edited journals "would, it seems to me, be one much more conformist and static, much less capable of progressive re-imagination or meaningful challenge to the powers that be." That is, of course, an empirical claim that requires empirical support. Is there evidence that peer-reviewed publications in sociology, criminology, economics, etc. are more "conformist and static, much less capable of progressive re-imagination or meaningful challenge to the powers that be" than student-edited law reviews? Shouldn't we demand some before we embrace this kind of claim? Perhaps it is legal academics that are mired in conservatism when it comes to their own professional norms.

Larry Rosenthal
Chapman

Alfred Brophy

Larry,

I'm confused by your first paragraph. Are you saying Stuntz' claims were made in a book? I'm guessing so, given that Schulhofer reviewed it in the Michigan book review issue. Certainly the book undergo peer review. Or are you suggesting that books get a different (and perhaps less rigorous) peer review than articles?

amused

I am with Rosenthal. The claim that peer review causes journals to be "conformist and static" is entirely empirical, and is entirely without support. In most disciplines, it is hard to make a career by writing stuff that's conformist and static. People in all disciplines have strong incentives to be novel and provocative. Peer-reviewed journals in all disciplines seek citation counts, and editors would rather publish a provocative citation-grabbing piece than a dull "we've all known this" one. As to Long's desire to keep academic work "capable of progressive re-imagination or meaningful challenge to the powers to be": even if we accept the questionable premise that leftist push is a legitimate purpose of an academic journal, all you need to do is look at sociology, anthropology etc journals -- peer-reviewed and full of "progressive re-imagination" to the core.

BoredJD

"How able are students to evaluate such claims (given their relative unfamiliarity with the conventional wisdom in the first place)?"

In my experience, the rigorous and time-consuming process of Googling the author's background and prior publication history, as well as any people that may have recommended the article.

All else being equal, an article is much more likely to get selected if it looks like it is publication ready. Don't leave cites for law students to fill in, have some RA bluebook it before you send it in, don't send it in with an unfinished conclusion.

Larry Rosenthal

Al:

The book was largely a recapitulation of a number of the author's articles. I don't know what kind of peer review process it went through prior to publication, if any, but from my familiarity with the "peer review" process for books of this type, it is nothing like the peer review process for peer-edited social science journals. "Peer review," of course, is not much of a help if it lacks appropriate rigor.

Larry

Alfred Brophy

That's interesting and I guess disturbing if we think that peer review for books is less rigorous than articles.

The comments to this entry are closed.

StatCounter

  • StatCounter
Blog powered by Typepad