Exempt Research & Expedited IRB Review: Curb Your Enthusiasm

A while back, over at PrawfsBlawg, Martin Pritikin had a useful post collecting advice for legal academics looking to break into increasingly popular empirical legal studies (ELS). As Jeremy Blumenthal notes in the comments, Step 1 is to be sure to get IRB approval. This post addresses what I’ll call, with a nod to Cass Sunstein’s work on Chevron deference, IRB Step Zero: Determine whether your research needs IRB approval at all.

Don’t worry, it’s an easy step: As Jeremy’s plenary admonition to all wannabe ELS scholars implies, the answer is almost certainly Yes. Although the regulations in theory establish three risk-based tiers of review — human subjects research (HSR) otherwise subject to IRB review that the regulations nevertheless exempt; HSR that is eligible for expedited review; and HSR that requires review by a fully convened IRB (everything else) — in practice, the first two tiers tend to collapse into the third. In this sense, and now I borrow from Matthew Stephenson and Adrian Vermeule, IRB review has only one step.

A quick note of clarification: As I’ve noted before (here and here), several projects I have in the works, beginning with Regulating the Production of Knowledge: Research Risk-Benefit Analysis and the Heterogeneity Problem, forthcoming next June in the Administrative Law Review, argue that we suboptimally regulate knowledge production. Just to be clear, my argument in that article doesn’t depend on my argument here about the broad scope of the regulations and their failed attempt to achieve risk-based levels of review.* Consider this post a public service for ELS types. That said, I draw here on The Heterogeneity Problem's background section, where interested readers will find the relevant citations.

Right. So, IRB Step Zero: Does your research require IRB review?

Whatev. My research isn’t federally funded. (In fact, it isn’t funded at all. Grrr.) IRBs are irrelevant to me.

As Justice Alito memorably mouthed, Not true.

It is true that, aside from research involving drugs, devices and so on that are under the FDA’s direct jurisdiction, by their terms, the relevant federal statute and regulations directly regulate only that HSR that is federally conducted or funded. In practice, however, a web of contractual relationships ensures that most HSR is subject to IRB review regardless of the source of funding, including virtually all HSR conducted by academics and their students.

When a faculty member (or any other institutional affiliate) receives federal research funding from any of more than a dozen federal departments and agencies, her institution contracts with the Office for Human Research Protections (OHRP) promising, in exchange for the funding, to ensure that the researcher submits her protocol for approval before a properly-convened IRB. There has been longstanding appetite in some quarters for directly federally regulating HSR, regardless of the source of funding. OHRP lacks the statutory authority to do that (and perhaps Congress lacks the requisite power as well). So instead, OHRP, in the contract of adhesion it executes with each institution (called a Federalwide Assurance, or FWA (pdf)), invites institutions to voluntarily promise to extend the regulations to all of their HSR, regardless of whether the federal government funds it or not. Virtually academic institutions have at least one federally-funded researcher, and between 75% and 90% of these, in turn, have agreed to extend IRB review to all HSR. (In July 2011, the Office of Science & Technology Policy and HHS issued an ANPRM that, among many other things, would make such extension mandatory.)

Of the minority of institutions that don’t contract with OHRP to extend IRB review, many nevertheless have adopted an institutional policy of extending IRB review to all faculty and/or student HSR. Similarly, many journals require that research submitted for publication have been approved by an IRB.

The bottom line? If a researcher isn’t subject to IRB review directly, through a federal grant or contract, she’ll likely be subject to it indirectly, through her institution’s contract with OHRP; through institutional policy incorporated by reference in her employment contract; or in her publishing contract. Thus is the oral historian, sociologist, anthropologist, memoirist, music theorist, education professor and, yes, legal academic swept down the rabbit hole of regulations.

But I’m not experimenting on people, I’m just talking to them.

You’re not engaged in human vivisection? Congratulations! But thanks to broad regulatory definitions of “research” and “human subject,” talking to people, analyzing existing data, and even observing people in public will often constitution HSR:

• “Research” is any “systematic investigation . . . designed to develop or contribute to generalizable knowledge.” (If you’d like to argue that you do not intend for your work to so contribute, feel free; it might work, and would be only moderately humiliating.)

• A “human subject” is any “living individual about whom an investigator (whether professional or student) conducting research obtains (1) Data through intervention or interaction with the individual, or (2) Identifiable private information.

• "Intervention includes both physical procedures by which data are gathered (for example, venipuncture) and manipulations of the subject or the subject's environment that are performed for research purposes."

• "Interaction includes communication or interpersonal contact between investigator and subject."

• "Private information includes information about behavior that occurs in a context in which an individual can reasonably expect that no observation or recording is taking place, and information which has been provided for specific purposes by an individual and which the individual can reasonably expect will not be made public (for example, a medical record). Private information must be individually identifiable (i.e., the identity of the subject is or may readily be ascertained by the investigator or associated with the information) in order for obtaining the information to constitute research involving human subjects."

Fine, my research is HSR. But it’s not risky; surely it’s exempt.

Not so fast. Many law professors — including, perhaps surprisingly, some who specialize in innovation policy or intellectual property, and even some whose empirical work should, under the regulations, be submitted to an IRB — know little or nothing about IRBs and their statutory and regulatory basis. Many others, of course, know quite a lot about IRBs.

And then there are those who know just enough to be dangerous. Such people sometimes suggest that regulatory categories for “exempt” research and research subject to “expedited review” (discussed below) significantly lessen the regulatory burden on relevant research(ers). As the Gershwins said, It ain’t necessarily so.

It’s true that the regulations provide for six categories of exempt research. But, as with most ex ante rules, there are significant limits as to how well-defined these regulatory categories can be. It requires an act of interpretation — of both the regulations and the study at bar — to decide whether a study is exempt. Notice that the regulations are silent about the allocation of this interpretive power. Nearly all institutions, in prudent adherence to agency guidance, require that it not be the researcher who decides that her project is exempt. (And, to be fair, if we trusted researchers' characterization of their studies, we wouldn't need IRBs in the first place.) Instead, researchers must submit any study they believe to be exempt from IRB review to . . . (wait for it) . . . the IRB.

At that point, typically, the IRB chair or another IRB member will review the protocol and recruitment plan to determine whether it is in fact exempt. So purportedly exempt studies don't necessarily undergo full-blown merits review by a fully convened IRB.

But, unless the IRB chair or her designee is really quite certain that the study is exempt, she will either deem it not exempt or send it to the full committee to determine its exempt status where, not infrequently, the exemption inquiry becomes difficult to distinguish from the risk-benefit analysis that takes place during full-blown “merits” review. (Why do IRBs tend to err on the side of more, rather than less, review? Because they're risk-averse, and because the kinds of costs they are most keen to avoid tend to be those associated with approving research that turns out to be dangerous or embarrassing or that incurs liability for the IRB and the institution, rather than the relatively hidden costs of blocking, substantively altering, or delaying welfare-enhancing research. That's the subject of its own post.)

Moreover, because the regulations constitute a floor, not a ceiling, even if an IRB determines that a protocol is exempt, it isn’t required to refrain from reviewing it. IRBs may — and regularly do — subject what are more accurately called exemptible proposals to expedited or even full IRB review. One study, for instance, found that 15% of the research proposals that had been reviewed by surveyed IRBs were exemptible, and that fewer than half of responding IRBs regularly exempted such exemptible research as analysis of existing data, interviews, and surveys. Indeed, some IRBs, by policy, simply subject all protocols to full review. As one commentator, himself an IRB member, put it: “There is no great gain in seeking [exempt] status.”

Well, even if the IRB reviews my work on the merits, it’ll expedite that review. No biggie.

Again, not so fast. Research qualifies for “expedited” IRB review if it is “minimal risk” and involves only one or more of 10 activities listed in the Federal Register (a list not updated since 1998). Under expedited review, the IRB chair or her designee can review the research proposal alone, which is often, but not always, faster than full review.

But, as with exemptible research, it is the IRB that determines both whether proposed research falls within an expeditable category and whether it involves “no more than minimal risk,” that is, whether "the probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests." IRBs are notoriously variable in their assessment of whether identical research procedures constitute minimal risk. (The Heterogeneity Problem discusses much of this empirical research on IRB variation.)

Moreover, as with exemptible research, IRBs “may,” but need not, expedite review of expeditable research. Given IRB risk-aversion, much expeditable research, like much exemptible research, in practice receives full IRB review. One study found, for instance, that of those high-volume IRBs surveyed, only 52% regularly conducted expedited review of studies involving a simple blood draw, and only 60% did so for studies involving non-invasive data collection from adults.

So, to return to IRB Step Zero: Does your study need IRB review? If it involves human beings and you hope that others might learn from its results, the answer is almost certainly Yes. Get thee to an IRB.

* What I call the (participant) heterogeneity problem in research risks and benefits would persist regardless of how much or little research the regulations govern (that is, regardless of how heterogeneous is the research subject to IRB review). In fact, I argue that intractable participant heterogeneity — combined with extremely sticky, if not quite intractable, IRB risk aversion — seriously frustrates attempts at risk-based research regulation. In The Heterogeneity Problem, I emphasize regulatory breadth not as a problem in itself (the article is agnostic about that), but in order to demonstrate the importance of IRBs to knowledge production and, secondarily, to help lay the foundation for skepticism about IRB expertise vis-à-vis this vast range of studies.

Cross-posted at Bill of Health.

Comments

[Glenn Cohen]

Great post Michelle! Curious whether you think law school ELS types are better of being reviewed by a single university-wide IRB or whether you'd advise them to consider pushing their dean to start a law school specific IRB (or together with other "low risk" departments"? Do you think some of the creep is "bleed" from the medical to the behavioral? Also are there any studies on heterogeneity of expedited review procedures specifically among IRBs?
Cross-Posted at Bill of Health

[Michelle Meyer]

[Note to readers: There has been quite a bit of discussion on this post over at Bill of Health. Parts of my answer to Glenn, below, cross-reference responses I've made to other comments there. Interested readers can find the fuller discussion at blogs.law.harvard.edu/billofhealth/2012/10/22/exempt-human-subjects-research-expedited-irb-review-curb-your-enthusiasm.]

@Glenn: There is, of course, lots that ails the IRB system, and so I say let a thousand flowers bloom as we try various ways of redressing these flaws. That said, I don’t think that non-biomedical IRBs are a panacea for much of what ails the IRB system. And so, given limited human resources, I’d rather see ELS types (and their colleagues) train their legal expertise on questions about the problems we’re trying to solve with the IRB system and how to optimize that regulatory system to solve those problems.

The regs were clearly designed by people whose primary work was in the area of biomedicine, and the regs reflect that origin in various ways. But short of amending the regulations to scale back that creep, there’s not much that can be done about what the regs call for across disciplines and methodologies.

No doubt, non-biomedical research is sometimes unduly delayed or restricted because, for instance, biomedical ethicists and researchers fail to understand such a study’s somewhat foreign aims or methodologies and apply the regs accordingly. But I’m skeptical that when IRBs delay or restrict social science or humanities research, they do so primarily because they’re reviewing non-biomedical as opposed to biomedical research. Many institutions (e.g., major research universities) already have an IRB dedicated to non-biomedical research, and my sense (again: we lack good data; see responses to other comments) is that these haven’t been a panacea at those institutions. I’d be happy and interested to hear otherwise from researchers.

Certainly, plenty of anecdotes exist in which an IRB shuts down social science research, and it turns out to have been a fellow social scientist on the IRB who hammered the final nail into the study’s coffin. Intradiscipline disputes about “proper” methodologies and sufficiently “important” research questions (not to mention turf wars and the usual petty academic politics) exist, and perhaps continue to grow with greater innovation in research methodologies, and IRB decisions can and do reflect these disputes, biases and limited understandings.

Conversely, although I’m skeptical of the popular view that biomedical research is riskier than non-biomedical research (such that we should restrict IRB review to biomedical research), to the extent that this is true (or simply believed to be true by IRBs), one can imagine biomedically oriented IRBs viewing the social science research they review as comparatively innocuous. If so, then biomedically-oriented IRBs ought to work in favor of social scientists hoping to get their work through relatively quickly and with few or no required substantive alterations.

Rather, I think that some of the biggest and most fundamental problems with the IRB system run across pretty much all types of research and all types of reviewers. The regs are necessarily open to interpretation, and IRBs tend to interpret both them and the studies they review in ways that involve more rather than less review. IRBs do so for a variety of reasons, including some I alluded to in my response to Paul Reitemeier. In addition to risk aversion (and here again, there are in turn multiple reasons for IRB risk aversion), IRBs, being comprised of human beings who have a natural desire to feel needed and useful, tend to want to spot at least some issues in a protocol (when was the last time a colleague asked you to provide feedback on a paper and you reported that it was just perfect as-is?). That very human tendency will persist even if it is fellow social scientists who are reviewing social science studies (indeed, the tendency may be stronger, as they are now “uber-experts” reviewing studies in their home discipline or field).

(On empirical studies of how IRBs deal with expeditable research, see my responses to Chris and Norm.)

[Marketing Research Companies]

I’d rather see ELS types (and their colleagues) train their legal expertise on questions about the problems we’re trying to solve with the IRB system and how to optimize that regulatory system to solve those problems.