Peer Review, ExpressO, And The Law Review Submission Process

As we slog through another law review article submission season - with authorial anxiety now painfully in view - I started wondering how Stanford's partial peer review process was coming along.  Rather than shout into the blogosphere I decided to call the Stanford Law Review. 

Masha Hansford, the new President,  tells me that while peer review isn't used in all cases, it is employed often - and more so this year than in the past.  She told me that, while the journal does rely on Stanford faculty, it also pushes articles to outside readers...relying especially on scholars cited in the pieces.  She said that in many cases, these readers provide fast turnaround.  She didn't want SLR to grab all the glory for the (partial) move to peer review, noting that both the Harvard and Yale journals use versions of it as well.  (Truth be told, many law reviews wisely refer selected article submissions to their own faculty for at least a coarse assessment of quality.)

Perhaps even more interestingly, Ms. Hansford confirmed something I've been suspecting for a while.  ExpressO is, at best, a mixed blessing.  After being buried by ExpressO submissions for several years, the editors at SLR realized that virtually every article they'd accepted did not come through ExpressO.  So the editors hiked the transaction costs of submission (ever so slightly) by asking authors to submit directly through the SLR website.  (Ms. Hansford actually told me that they insist on this method, though the website does not say so.)  As a result, Stanford has seen a dramatic drop in the number of articles submitted.  

In reality, ExpressO has had a much greater impact on lower profile journals.  Even in the age of the Post Office, most authors sent drafts to Stanford.   But back when you had to stuff real envelopes, you'd limit your mailing. Now many authors target well north of 100 journals.  This, in turn, means that virtually every law review is paralyzed by volume.  While the law review submission system has always been dubious as a quality filtration mechanism, at least you had the sense that the expedite system could  have the effect of helping better articles bubble up.  That process is a lot tougher if the editors of the Oopsy Doopsy Law Review are now staring down 1000 submissions.  At some point, and we may have already be there, the whole system will seize up. 

ExpressO could protect its brand by adopting a LexOpus strategy: regulate the  number of submission by each author.  Otherwise the overload will cause more journals to create submission roadblocks.  And the first logical barrier is a ban on ExpressO.

Comments

[James Grimmelmann]

1000 submissions at Oopsy Doopsy? Is the number really that low?

[Scott Boone]

In the past, we (speaking very broadly) have speculated on the sort of proxies law review editors use (intentionally or unintentionally) to assist them in assessing quality. Paralyzing levels of volume would seem likely to increase reliance on such proxies.

It would be interesting to see if we could find out from editors if they are developing methods of dealing with a slightly different problem relating to high volumes of submissions. How do they choose which articles to put through their review process first? Is it done strictly based on submission dates - first submitted, first reviewed? Do interesting topics or titles grab attention and thus an earlier review? Do previously discussed proxies (c.v., prior publications) come into play again?

[Ben Barros]

Some sort of limit might help with the collective action problem that hurts any prof who voluntarily cuts back on the submissions. But BE Press has an obvious financial incentive to not limit the number of submissions. There are some other problems with limiting submissions, especially for untenured folks and people who want to go on the market. These folks have some time pressure on getting an acceptance, and would have a high risk of a failed submission.

If technology is increasing the number of submissions, might law reviews need to adapt? I realize that it might be problematic to put the burden entirely on the law reviews, but I’m not sure there is a workable solution on the law professor end. I'd hate to see them rely even more on the proxies that Scott mentions. Law reviews could try to review submissions year round, so that the submissions come in at a more manageable pace. Knowing the way that most law professors think, the top 15 or so law reviews would need to make this change for it to really impact professor behavior. Or, law reviews could enlist more of their staff to do quick initial reads of submissions. I know that some journals already do this, but many place almost the entire review burden on a few articles editors.

[Scott Boone]

Two more thoughts on addressing volume problems (followed by two radical suggestions):

1. Howard Wasserman has a post today at Prawfsblawg about journals offering longer decision times with their offers, but only if the author agrees to expedite only at substantially higher ranked schools. I mentioned in the comments that, if universally adopted, it might eliminate "lateral" expedites undertaken not out of an author's preference for that placement but out of a desire to extend the time for expediting at higher ranked journals. If this sort of lateral expediting is significant in terms of volume, then it's elimination could ameliorate some volume problems.

http://prawfsblawg.blogs.com/prawfsblawg/2010/03/law-reviews-playing-the-game.html

2. I would guess that it might only make a small difference, but I have often wondered how well the withdrawal system works, on both sides. How many professors withdraw their submissions after accepting an offer and how many journals actually keep track of withdrawals? I primarily wonder because I have gotten responses (both acceptances and rejections) from journals long after I have sent the journal a withdrawal notice. Are journals reviewing articles that are no longer available?

Two radical suggestions:

1. Revive the idea tossed out by Ann Bartow in the Law School Moblog a couple of years back over at Madisonian: Just have faculty members publish in their own school's journals. (I can't remember if that was the full scope of the suggestion, or whether Ann was endorsing it or merely "tossing it out there." I just want to be careful I'm giving credit while not misrepresenting what she said.) That would be a game changer in many, many ways.

2. Have journals follow the same approach suggested for the elimination of spam. Have journals charge for review. It would have many negatives, including those for untenured and on the market folks mentioned by Ben. (Interesting metaphor: mass submissions as spam, proxies as spam filters, etc.)

[Ben Barros]

Another possible step in the right direction, related to Scott's first point above: journals could offer to accept or decline within a certain time period (perhaps 3 weeks) if the author agrees not to expedite (or agrees only to expedite to certain highly-ranked reviews). This might help with some of the volume problem at some lower-ranked journals - they could devote their reviewing time to submissions that they could actually get, and could put the submissions that might just be interested in shopping up at the bottom of the stack. I think a lot of professors would make this offer to journals, but right now there is no effective mechanism to do so -- a promise in a cover letter only works if it gets read. Maybe Expresso could add a new category or categories of submissions?

On a related point, I can't stand it when professors submit to lower-ranked journals hoping to shop up, and then decline an offer because it isn't good enough. I'll give a pass, maybe, to pre-tenure or on the market folks. But for everyone else, this is really bad conduct.

[Christopher Yoo]

Word is that electronic submission is having a similar effect on the clerkship application process. Rather than making the process more merit based, lowering the cost of applying has dramatically increased the number of applications each judge receives. As a result, judges are relying more than ever on personal contacts, letterhead, and other such proxies to serve as an initial screen.

[Tim Zinnecker]

Ben,
As to the point in your second paragraph, I'm curious why you "give a pass" to the pre-tenure or on-the-market folks. "Really bad conduct" (and I agree with your term) is no less "really bad" depending on the actor, is it? Or are we (and I'll include myself in that term) suggesting that those particular actors have a (defective) market-driven reason to place well (all this on the tainted assumption that the higher the placement, the better the quality of the work)?

[Rick Paris]

The problem isn't with law reviews. The problem is with law professors. You guys are acting as if this odd process results in good pieces going unpublished. It doesn't. The result here is that the best articles aren't always placed in the best journals. So what? How about law professors judge the quality of an article on its own merits?

[Scott Boone]

As Rick points out, most of the elements of the problem are "tautological" in the sense that placements in high ranking journals are valued by law professors as authors because placements in high ranking journals are valued by law professors as members of P&T committees, as hiring committees (of laterals and entries), as consumers (in choosing what to read and cite), as deans trying to boost reputation, as peer voters in USNEWS ballots, etc.

Maybe we need to get our own story straight.

[Ben Barros]

Tim, yes, the reason why I might give a very junior person a pass here is that (a) they have a [deeply flawed] market-driven reason to place well, and (b) that they might not have other proxies yet that would get them a read at a top law journal without the expedite. But, as I said, I _might_ give them a pass. You are correct that the conduct would still be bad.

Rick, I think that you are generally correct. It is frequently observed that law journal placement has little correlation to quality, and instead often correlates most strongly with various status-based proxies. Here's the problem -- almost all law professors know this, at least to a certain degree, but they don't internalize it in their behavior. For hiring, tenure, promotion, and general status in the profession, high placements matter a lot. They shouldn't but they do. Even if it is aimed at the symptom, rather than the disease, some sort of management of the submission process is much more likely to actually happen than a radical change in the views and behaviors of the professoriate.

[James Grimmelmann]

I agree with Rick, but would point out that the crazed system we currently have also does a disservice to the students who are supposed to be learning something from their work on law reviews. Overwhelming volumes of submissions make their engagement with scholarship more superficial.

[Bridget Crawford]

I'd be curious to hear from authors and editors about experiences with an "exclusive-submission-in-return-for-fast-read" policy. Any data to suggest that such a policy helps journals secure articles that they otherwise would "lose" to "better" journals? Or that the policy helps professors secure "better" placements than they might get under the current system?

[Ben Barros]

Bridget, I doubt there is any data because I don't think that any law reviews offer that policy. Duke did (and maybe still does) offer quick consideration in return for an agreement to only expedite to at most four other journals. I might be wrong, and would love to hear about any journals that do.

[Howard Wasserman]

A couple of journals (Georgetown and someone else) used to (and perhaps still do) some version of exclusive-submission during summer (June and July), right up until the opening of the fall cascade.