Search the Lounge


« Nebraska: Hiring Needs | Main | Archival Research: There's An App For That »

September 15, 2010


Feed You can follow this conversation by subscribing to the comment feed for this post.


I've never seen any evidence that impact tracks media appearances or even blogging (I blog as much as anyone, and less than 10% of my citations are to my blogs). Where did that idea come from?

Jacqueline Lipton

Brian: Thanks so much for commenting. I must admit that I've not reproduced/investigated the part of Sisk's study that relates to my school (or any other school), but I was going on my knowledge of the scholarship of the people at my school noted as amongst the most cited faculty. No doubt we have all written law review articles, but several of the people included in the list have not written traditional law review articles either very much or very recently - not that the age of the article should necessarily be relevant to how often it has been cited. I simply noticed that a number of our most cited faculty have done significantly more in the way of media and blogging than in the way of traditional law review articles, particularly over the last 5-10 years. I also noticed that some of our highest profile "traditional scholars" who don't really do much/any blogging or media did not show up as being highly cited. In other words, this is my own anecdotal impression based on the way our school was ranked in Prof Sisk's rankings. I may well be incorrect in my assumptions - indeed I kind of hope I am. But it would be interesting to know more. Maybe I'll have to take a closer look at the actual citations and see if I'm right or not. (And I'm very interested to hear that less than 10% of your citations are from blogging - that assuages some of my concerns about impact rankings.)

Dan Markel

Yup, much as it chagrins me to say it, I'm with Brian on this :-)

Jonathan H. Adler

Jacqui --

I'm with Brian as well. Instead of speculating, why don't you replicate a few of the study's searches. I think you'll be surprised.


Jacqui Lipton

I'm so sorry - I didn't mean to offend anyone personally or collectively. I thought that it was OK to speculate on a blog and I'm happy that people have responded with their views on their own citation counts. Some others obviously have much more time than I do to be thinking about this. As I said, I was just musing and would be very happy to be incorrect. However, I do have an uneasy feeling about law professors always having to count everything and put so much weight on these kinds of ranking exercises. Many of us criticize the U.S. News rankings, but is it really the solution to do more or different rankings? Maybe it is for all I know. And of course I personally benefit from Sisk's study because I count as having a lot of citations so if you are all convinced of the usefulness and meaningfulness of these rankings, then it's great for my scholarly reputation too.

Mary Dudziak

There is another point to make about these sorts of rankings. But first I wanted to say: as much as I agree w/ Brian, Dan and Jonathan that Jacqui's questions are best answered by the data, I don't think Jacqui needs to apologize for raising questions. And frankly, we're not all number crunchers. (Without making assumptions about Jacqui's skills in this area) a non-quantitative person shouldn't feel that they can't engage in a discussion about quantitative data just because they're not going to be good at running the numbers. If Jacqui's questions are answered by data -- great to know. Please share the results.

But the point I want to raise is different. This week one of my smartest and most productive colleagues just had a paper accepted for publication in a published collection of essays in the field of social psychology. The scholarly impact of this kind of paper will never be accurately registered in citation count studies confined to legal databases.

Let me say that I agree with the sentiment behind these studies, as Fisk, et al. put it: "As legal scholars, we write for an audience. It is right and appropriate, then, to ask whether anyone is listening. And, if possible, we should answer that question by something more reliable than anecdotes, past accolades, or casual assurances by those in our close circle that they’ve read this or that article." The question is what that audience is: Legal scholars alone, or the academy as a whole? Legal scholars are part of universities, and a scholarly discourse that extends beyond our schools, and ideally beyond our national borders. These studies are limited to a legal citation database that does not capture this sort of impact.

Leiter and Fisk et al. acknowledge limitations, including the impact on interdisciplinary work, but suggest that “an imperfect measure may still be an adequate measure, and that might appear to be true of citation rates as a proxy for impact as a proxy for reputation or quality.”

But the numbers have a tendency to take on a life of their own, and as they are circulated the caveats tend to drop away, and we are left with the numbers themselves, as if they were a representation of something true, rather than one imperfect measure that perhaps necessarily has systematic flaws.

Jonathan H. Adler

Jacqui --

No offense taken. There's certainly ample reason to speculate about the value of rankings, including this one, and the desire to count or rank everything. In this case I do think this sort of study provides a meaningful, though hardly perfect, measure of the recent influence of a scholar's past work within the legal academy.

Without question a citation study like this one will reflect the subject matter bias of law reviews, so highly productive scholars in some areas (e.g. tax) are less likely to be recognized for their contributions, and Mary is absolutely correct that some important inter-disciplinary work is likely to get slighted by this methodology.

Citation count studies will tend to reward past work more than present work. So if there is someone who used to be a productive scholar, but gave it up for blogging in 2008, we shouldn't expect to see the effect of that in a citation count study that looks at, say, the 2004-2010 period. Similarly, someone who wrote a seminal article on a particular question many years ago may still receive lots of citations today regardless of what they are doing today.

As for blog citations, my experience is like Brian's. Fewer than 10 percent of my citations are to blog posts or media citations, and most of these citations are to posts that relate directly to my scholarship.


Jacqui Lipton

In light of these discussions, I'm wondering if it's worth having a separate discussion on the relationship between blogging and scholarship. I happened to be chatting about that question this morning with some colleagues - and Jon raised another issue in his last post as well about this by saying that most of his blog posts that are counted in the citation studies relate to his actual scholarly articles rather than other issues. Another colleague suggested to me that people who blog a lot may actually be given more notice as scholars because their audience is simply more familiar with who they are and what they have to say. I thought that was also an interesting point. I also have other colleagues who think that blogging detracts from scholarship because it takes time away from time you could be writing law review articles. I don't personally agree with this view, and I often think blogging informs scholarship and vice versa. (And the "time management" argument really makes certain assumptions about how much time a person actually has for any activities outside the classroom and the family.) I recently constructed a law review article largely based on discussions I had about a particular issue on another blog and I too have cited others' blog comments in scholarship - so I think it's an interesting symbiotic relationship and one that would be worth discussing more. And I'm sure it has been discussed on other blogs as well. I'd be happy to receive links to relevant discussions.


Just as successful blogging gets an author SSRN downloads (is there really a debate about this?), it doesn't take too much extrapolation to see how blogging can translate into citations. Not citations for blogposts but citations to the articles that get more attention because of the author's status as blogger and the traffic directed to those articles that would otherwise not be read or seen. Yes, yes: test away if you have nothing better to do with your time. But this is reasonable speculation, I'd imagine.

Dan Markel

Jacqui, I didn't take offense and didn't mean to give any (by suggestion of chastisement) either. My apologies if I seemed piqued. Just genuinely puzzled at the initial hypothesis. For what it's worth, my blog posts have generated I think no more than 2% of my cites. The previous anon comment, on the other hand, has a ring of plausibility.

The relation between SSRN and blogging is also interesting. A number of us perma bloggers at Prawfs, Co-Op or here at Faculty Lounge do not have unusually high SSRN downloads despite what I think of as very impressive scholarship track records. And in the case of some, that's the case notwithstanding some efforts to furnish links to the work. In any event, let a thousand flowers and theses bloom!

Jacqueline Lipton

Not at all, Dan. And interesting re the idea that many bloggers don't have unusually high SSRN citations. Someone's next law review article, perhaps?

The comments to this entry are closed.


  • StatCounter
Blog powered by Typepad