Will Claude AI Revolutionize the Law Review Article Selection Process?

I continue my experimentation with various large language models and AI tools to understand how they might be used to support legal scholarship. Lately, I've been playing around with Claude, a rival of Chat GPT.  (I have no relationship with either company.) It's free to use Claude's entry-level model. I have not paid for the upgraded version, which costs approximately $20/month.  ReadWrite has an informative review of Claude here.

I recently asked Claude to "summarize and evaluate the strengths and weaknesses" of the abstract and introduction of one of my accepted-but-not-yet-published articles, Taxing Sugar Babies, Minn. L. Rev. (forthcoming 2024). (For whatever it's worth, I'm trying to promote my paper per se, but because I wrote it, I can more readily determine how accurate Claude's results are.)

Results after the fold. TLDR; Claude's summary was pretty good and its evaluation of the weaknesses of the abstract and intro were fair. I suspect that student law review editors (either with or without being prompted to do so by Scholastica, etc.) will be integrating AI into the article submission process in the near future.

Here's the specific prompt I entered:

"Here is the abstract and introduction of an academic article. Summarize it and evaluate its strengths and weaknesses."

I then pasted in the material I wanted Claude to evaluate. I would have preferred to upload the full article for evaluation, but I didn't have that option with the free plan. With both the free version and the paid plain, Claude does impose quota limits that reset each day.  (That is a decided negative, for my purposes. My paper is less than 14k words with footnotes, but it was too big for Claude.)

Here's what Claude replied:

Here is a summary of the abstract and introduction:

This academic article examines how "sugar babies" (typically younger women) and "sugar daddies/mamas" (typically older, wealthier individuals) discuss taxes and their tax obligations in online forums related to their "sugaring" relationships. These relationships often involve the exchange of cash, gifts or other benefits from the sugar daddy/mama to the sugar baby in return for companionship that frequently includes a sexual component.

The article makes three main claims:

1. The dominant discourse is that sugar babies receive untaxable "gifts" rather than taxable income for companionship.
2. This framing of payments as gifts is likely incorrect from a tax law perspective, but authorities are unlikely to prosecute sugar babies for not reporting the income.
3. The "gift" framing helps maintain the gray area between personal relationships and commercial sex work, devaluing the labor of sugar babies and perpetuating stigma around sex work.

The introduction provides background on the history and nature of sugaring relationships, positioning them between unpaid companionship and clear-cut sex work like prostitution. It argues that examining tax discourse in this area can reveal attitudes and stigma around sugaring and sex work.

Strengths:

• • Novel application of tax law analysis to the sugaring phenomenon
  • Synthesizes sugaring's blurring of boundaries between relationships, sex work, employment
  • Connects tax framing to broader narratives and stigma around women's labor
  • Interesting qualitative approach analyzing online discourse

Weaknesses:

• • Methods and data sources (Reddit forums) could be further justified
  • May need more theoretical framing around sex work, sugaring, feminism
  • Stronger positioning of the study's broader implications would be helpful

Overall, the premise is intriguing and well-motivated. With a rigorous analysis and positioning of the study's significance, it seems potentially publishable, especially given the emerging research around sugaring relationships and online qualitative methods. Further describing the study's theoretical groundings and implications could elevate the strengths.

I think that Claude's summary of my article is accurate. I was impressed that Claude was able to extract the three "moves" of the paper (which I did identify as I was writing, to be fair).  I also thought that Claude's analysis of weaknesses was spot on, as well. I do need to better explain the study's methods and implications. A more explicit theoretical grounding will strengthen the paper, too. This is actionable feedback that I can incorporate into the next draft.  That's a good use of this technology and consistent with using AI to assist in improving the quality of one's scholarly work product.

Thinking to the future, I can easily imagine that Claude or a similar tool could be helpful for student law review editors who must plow through hundreds of article submissions each season. Note that, unlike Chat GPT, Claude refrains from using prompts or responses for training (more here). Otherwise, I would think that there could be ethical concerns using AI for the same tasks, if the law review did not secure the authors' consent to upload the submission for evaluation by a large language model. Perhaps such a consent via a radio button will become part of the standard submission process, a metaphoric "price" that authors pay to submit to a particular journal or even to use Scholastica, etc. (apart from the already hefty dollars-and-cents price tag that Scholastica imposes!).

I can imagine a business like Scholastica joining forces with an AI company to provide student editors with tools to summarize papers, evaluate them for novelty, map ways that a submitted article connects with prior scholarship in the field, etc.

This is something I will continue to watch with interest.

 

Comments

[Alex]

I understand this seems to have "worked" well for your paper, as you surely have experienced in workshops, other people (or AI I suppose) might give you conflicting advice about what the positives and negatives of the paper are. This is the nature of scholarship.

Setting that side, do you have any confidence that this would be broadly the case? You might want to consider a caveat of the small-N problem. I have not used this platform, but having seen the pervasive falsehoods and many problems with others, I would need to see many more examples before even thinking this would be a reasonable tool. I know you are not necessarily advocating for this, but it seems like an obvious implication from your quick blog post saying it "worked" in this instance.

[Al]

This is great. Astonishing but great. One question. How much to you think it's going to "novel application." ("Novel application of tax law analysis to the sugaring phenomenon.").