At an AALS Annual Meeting panel on the electronification of law libraries over the weekend, an interesting point was made about the need (or otherwise) for peer reviewed law journals. A commentator at the session noted that he could see the point of peer review in medical and scientific journals for the purposes of checking research methodologies and conclusions, but did not see the point in law/humanities journals because it just fueled an "old boys' network" attitude to publication and there was no need for checking methodology etc in non-scientific fields.
I'm not 100% sure that this is correct. I've worked in countries with peer-reviewed law journals before moving to the U.S. and I do think that peer review serves a function with respect to law journals, particularly as compared to review by law student editors. Many have noted in the past that student editors are not necessarily best placed to check the accuracy and/or novelty of new scholarship, particularly in specialist fields with which they may not be familiar - although hopefully many editors faced with such scholarship will seek expert opinions from faculty members. While I think there is perhaps some merit in the points made at the panel, I suspect it's an overstatement to say that peer review wouldn't add anything to our current system in the U.S. But I'm also loathe to say that it's a perfect answer to all the problems that journals face in article selection.
A review of the literature on peer review reveals it to be largely a random process.
Posted by: anon | January 11, 2010 at 10:32 AM
I'm somewhat less negative about law reviews as opposed to peer-review journals than are some, but the comment about an "old boys network" seems completely backwards to me. If it's done right, peer review for a journal is also blind review. There are different degrees of blindness, and in some (maybe most- I'm not sure) case editors know who the authors are even though referees do not. Since editors are very important in the selection process (they often make a first-cut assessment as to whether something gets reviewed at all, decide on whom to send the article to [some reviewers are reliable rejectors of articles, some will be either hostile or friendly to a view, etc.], and so on), there's still a chance for an "old-boys network" or a "big-name effect" to play a role. I'm pretty sure that almost everything by big names gets sent out for review, regardless of quality, while the same isn't true for the unknown person. But this is at least reduced greatly when, as is more and more the case, the first editorial work is also done "blind". But this is all the opposite with law reviews, where names and often CVs are attached and obviously (and explicitly) used as proxies for quality. In such a case the "old boys network" is obviously much more likely to have an impact. So, I don't see how the remark isn't completely backwards of reality. Is there something I'm missing here? (My impression is that the limited peer review put in place by some law reviews isn't blind, but I'm not sure of that. Obviously peer review needs to be blind review if it's to gain all of its benefits, but making this possible would require changing the citation norms and the like in submitted articles for law reviews, among other things.)
Posted by: Matt | January 11, 2010 at 11:48 AM