Over at Violent Metaphors, Jennifer Raff (Anthropology, Texas) offers her thoughts on How to Become Good at Peer Review: A Guide for Young Scientists. Here are some of the basics :
- "Ask yourself honestly whether this paper falls within the scope of your expertise. If it falls too far outside of your discipline and knowledge, it’s better to leave the review to someone else."
- "If you don’t think you can commit enough time to the review in order to make the deadline, you should decline."
- "Absolutely do not review a manuscript if you have a strong feeling (positive or negative) about any of the authors."
Read the full post here. Rabb goes on to offer suggestions for specific questions a reviewer can answer:
- "Is the research original and interesting?"
- "Is the current state of knowledge about the field accurately represented?"
- "Can you think of a better way to address the research questions than what the authors did?"
- "Are the authors' interpretations backed by the data and convincing?"
It strikes me that much of this guidance is useful to those who are invited to evaluate a national colleague's scholarship in connection with another school's tenure process. To be truly useful, a tenure letter has to be a frank evaluation of the piece (or pieces) under review, not a statement about the reviewer's methodological (or even subject-matter or ideological) preferences. To critique a piece for being, say, critical scholarship instead of empirical scholarship is decidedly unhelpful. So, too, is summarizing the arguments an author makes and saying that they are "impressive" or "unimpressive." A really good tenure letter should provide context for the scholarship under review and evaluate its success or failure on its own terms.