Facebook Announces New Research Policies

A WSJ reporter just tipped me off to this news release by Facebook regarding the changes it has made in its research practices in response to public outrage about its emotional contagion experiment, published in PNAS. I had a brief window of time in which to respond with my comments, so these are rushed and a first reaction, but for what they're worth, here's what I told her (plus links and less a couple of typos):

There’s a lot to like in this announcement. I’m delighted that, despite the backlash it received, Facebook will continue to publish at least some of their research in peer-reviewed journals and to post reprints of that research on their website, where everyone can benefit from it. It’s also encouraging that the company acknowledges the importance of user trust and that it has expressed a commitment to better communicate its research goals and results.

As for Facebook’s promise to subject future research to more extensive review by a wider and more senior group of people within the company, with an enhanced review process for research that concerns, say, minors or sensitive topics, it’s impossible to assess whether this is ethically good or bad without knowing a lot more about both the people who comprise the panel and their review process (including but not limited to Facebook's policy on when, if ever, the default requirements of informed consent may be modified or waived). It’s tempting to conclude that more review is always better. But research ethics committees (IRBs) can and do make mistakes in both directions – by approving research that should not have gone forward and by unreasonably thwarting important research. Do Facebook’s law, privacy, and policy people have any training in research ethics? Is there any sort of appeal process for Facebook’s data scientists if the panel arbitrarily rejects their proposal? These are the tip of the iceberg of challenges that the academic IRBs continue to face, and I fear that we are unthinkingly exporting an unhealthy system into the corporate world. Discussion is just beginning among academic scientists, corporate data scientists, and ethicists about the ethics of mass-scale digital experimentation (see, ahem, here and here). It’s theoretically possible, but unlikely, that in its new, but unclear, guidelines and review process Facebook has struck the optimal balance among the competing values and interests that this work involves.

Most alarming is Facebook’s suggestion that it retreat from experimental methods in favor of what are often second-best methods resorted to only when randomized, controlled studies are impossible. Academics, including those Facebook’s statement references in its announcement, often have to resort to non-experimental methods in studying social media because they lack access to corporate data and algorithms. “Manipulation” has a negative connotation outside of science but it is the heart of the scientific method and the best way of inferring causation. Studies have found that people perceive research to be more risky when it is described by words like “experiment” or “manipulation” rather than “study,” but it’s not always the case that randomized, controlled studies pose more risk than do observational studies. The incremental risk that a study — of whatever type — imposes on users is clearly ethically relevant, and that's what we should focus on, not this crude proxy for risk. I would rather see Facebook and other companies engage in ethical experiments than retreat from the scientific method.

It’s also unclear to me why guidelines require more extensive review if the work involves a collaboration with someone in the academic community.

Comments

[Enrique Guerra-Pujol]

Facebook's self-serving statement still misses the point: its social-science experiments, however useful, are non-consensual

[Enrique Guerra-Pujol]

Why aren't my comments showing up? I wrote: Facebook's self-serving statement still misses the point: its social-science experiments, however useful, are non-consensual

[Steve Wilson (@Steve_Lockstep)]

In trying to get Facebook to embrace scientific method, the dilemma is that some human experiments just should not be done.

Facebook previously claimed that "informed consent" for the mood experiment derived from a fragment of a sentence in their Data Usage Agreement. But we all know what consent forms in properly constituted human research experiments look like. They go to great pains to explain what's going on, to leave the subject in no doubt whatsoever as to what they're signing up for. On the other hand, the infamous mood experiment design actually depended on subterfuge. And it was never proper science but a barefaced exercise in product design; one of the authors Adam Kramer said as much: "[We] were concerned that exposure to friends’ negativity might lead people to avoid visiting Facebook".

The real test of Facebook's commitment to research rigor will be whether they actually come to reject any experiments because proper consent cannot be obtained.

See also http://lockstep.com.au/blog/2014/06/30/facebook-lab-rats

[Ray Campbell]

I think one of the interesting things going on here is the collision of two different paradigms with regard to testing and observing behavior. One is the paradigm of academic research. The other, and there is a lot more of it out there these days than there is of academic research, is the paradigm of direct marketing. Back in the day direct mail guys would test A versions versus B versions of direct mail solicitation letters, seeing which would generate a stronger response. The winner would become the new A version, and would be tested against a slightly modified version that became the new B version. When the internet arrived, it allowed more or less instantaneous A/B tests. Page design, marketing messages, site design, emotional appeals versus logical appeals, people shown with products versus just the product photos, happy people shown with the products versus serious people shown with the products, etc., would all be tested against unknowing visitors to the site, seeing which generated a more desirable response.

What Facebook did that distinguishes them from every other direct response internet marketer out there is that they embraced some, but not all, elements of the academic paradigm instead of staying within the proprietary, largely secret marketing paradigm. I would be astounded if other marketers, in order to get information that will help them make more sales, are not running studies just as intrusive as Facebook's but not telling anyone what they learn.

As a practical matter, marketers cannot be held to academic standards without very intrusive regulation. Website operators will in the normal course tweak what appears on their site, and will in the normal course run statistical analysis on the site logs to see how people respond to the changes. That's an invisible and inevitable part of sites that change incrementally and constantly. You would really have to impose some pretty strong regulation on how site owners are allowed to modify and track their sites in order to get a handle on that, and such regulation would raise legal and ethical issues of its own.

Now that Facebook has put the two paradigms on a collision course, it's going to be interesting to see how it plays out. I would be that it's going to be a good long while before any other site admits to testing of the kind that Facebook has done, but about a nanosecond before some site carries similarly invasive tests.

[Michelle Meyer]

Thanks for the comments. I'm about to head into a board meeting at the ungodly hour of 8am on a Saturday, so my response will have to be brief(ish) for now.

Enrique, your comments don't show up because Typepad has the Worst. Spam filter. Evah. It appears to have two modes: let everything in (in which case we're inundated with spam) and let nothing in (in which case even my own comments on my own posts on my own blog while signed into Typepad get stuck in the filter). So don't take it personally. For the same reason, I've only just now retrieved Steve's comment. So, for all commenters: when in doubt as to why your comment has not shown up, please lend the poster the charity of assuming, absent some evidence to the contrary, that it is Typepad's fault.

Now on to substance. Enrique and Steve, yes, as I have said from the beginning, this particular FB experiment did not involve informed consent. However, as I've also previously noted, it's quite incorrect to suggest that each and every experiment requires, either as a matter of U.S. federal law or as a matter of any remotely sensible ethics, rigorous informed consent in which investigators "go to great pains to explain what's going on, to leave the subject in no doubt whatsoever as to what they're signing up for." Zach Schrag puts this in historical context here: http://www.institutionalreviewblog.com/2014/06/a-bit-of-historical-perspective-on.html.

For our law readers, since much (not all) of the backlash against Facebook seems to be regrettably mixed up with criticisms of Facebook's other policies (ones that actually affect informational privacy, as this experiment did not), corporate data practices in general, and/or broader push back against Corporate America, let me offer an example of ongoing research closer to home that I suspect most law profs are strongly supportive of: The Implicit Association Test (IAT). The short online consent form, which subjects are free to click through without reading in any case, discloses merely that IAT studies “examine your ideas, beliefs, and opinions about different topics. You will answer some questions and take an IAT in which you will sort words into categories as quickly as possible.” There is no mention of the purpose of the study, which is to investigate whether the subject holds highly stigmatized “ideas, beliefs, and opinions” like sexism and racism (or, more precisely, harbors associations that may or may not reflect such “ideas, beliefs, and opinions”). Nor does the consent form make a single mention of any risk to subjects of participating in the study, such that they may come away with the impression (correct or not) that they are implicitly sexist or racist. The reason why researchers do not seek subjects’ fully informed consent, and why IRBs have approved this research anyway, is that IAT results would likely be biased if subjects were told up front the kinds of associations investigators are looking for, and learning about implicit attitudes is pretty important.

To be clear, I’m not saying that the IAT is on all fours with the Facebook-Cornell study. Although I actually think that the IAT imposes more risk on subjects than did the Facebook study, IAT subjects are at least told that they are being invited to participate in a study, and that they can withdraw at any time by closing their Internet browser window. My narrow point for now is this: to consent to take the IAT is not to give consent that is fully informed and it is, therefore, not to voluntarily assume the (wholly unstated) risks of participating in that study. Compared to the elements of consent normally required by 45 CFR 46, the IAT consent form is extremely pared down. I frankly think there is some room for improvement here, and especially in its online debriefing form, which among other things neglects to tell subjects receiving the potentially upsetting (and easily misunderstood) information that their results show implicit bias that many, many subjects’ IAT data also suggest similar associations, including women (my own results showed the strongest level of “sexist” implicit association the IAT reports and I won the UVa-wide Zora Neale Hurston Prize for best Studies in Women & Gender essay in grad school, dammit), members of racial and ethnic minorities, and so on. But my point for present purposes is that IAT subjects’ consent is not the fully voluntary, informed consent that is required for riskier research and ideal even for minimal risk research and that many members of the public (and some academics, especially those familiar only with biomedical research) seem to believe is always ethically and legally required.

In short, although I am one of the biggest proponents of autonomy you’re likely to find within the field of bioethics, autonomy is not the only value worth preserving. We should and do care about human welfare, too, When the best available evidence just prior to a proposed study suggests that its marginal increased risk is similar in size and likelihood to the risks subjects encounter in daily life, and when it could not otherwise practicably be done, a study may qualify for an alteration or even waiver of the elements of informed consent that riskier studies require. This is both a regulatory provision and a reasonable ethical principle that seeks to balance liberty and welfare rather than fetishizing one over the other.

Of course, in those cases where fully informed consent is not obtained, the risks and potential benefits of a study become especially important, and since researchers have incentives to overplay the potential benefits and overlook the risks of the work on which their careers depend, it is helpful to have a neutral third party conduct risk-benefit analysis, and I've suggested from the beginning that Facebook ought voluntarily to have sought such outside review as well as debrief subjects. (Unfortunately, IRBs are saddled with their own biases that distort risk-benefit and other analysis. But I digress.)

You are of course free to reject that trade-off as a matter of ethics (as opposed to law), but you’ll have to justify why, when we impose costs on each other constantly (often serious and for the most frivolous of reasons) without either informed consent or prospective licensing of those activities (by local regulators who lack even an ounce of the characteristics we generally demand in the administrative state and elsewhere, such as transparency, accountability, researchers’ right of appeal and other due process rights that constrain IRBs’ often capricious and arbitrary decisions), that we should make it so difficult to impose minimal risks in the pursuit of rigorous evidence that cannot be had easily or at all in any other way. And you'll have to be prepared to disavow years and years of psychology research, including a fair amount that forms the basis for behavioral law and economics.

Ray, as for the "pretty strong regulation on how site owners are allowed to modify and track their sites . . . that would raise legal and ethical issues of its own," see https://medium.com/@JamesGrimmelmann/illegal-unethical-and-mood-altering-8b93af772688

[Ray Campbell]

If Professor Grimmelmann is right, and if what he means to say is that the Common Rule or state analogs apply to A/B testing by marketers, I stand by my original point that it's going to get very interesting as two paradigms collide.
Running an internet website involves pervasive observation of the behavior of visitors to the site through the site logs. Improving an internet site, along whatever metric you want to improve it, involves presenting randomized samples of visitors different experiences and tracking how their responses vary. Life online involves constant observation and manipulation of behavior.
Most internet marketers would not think of their analysis and manipulation of site visitors as akin to academic research. It's just what they do. They view it as making their internet site more responsive and effective. They think that even though, at a certain level of abstraction, the manipulation of human visitors to websites does indeed bear a resemblance to the manipulation of research subjects, although the goal is more often to make people feel cupidity or engagement rather than happiness or sorrow.
It's not clear to me how website operators respond. For pure content sites, where there may be only a browserwrap rather than a clickwrap terms of use contract, I'm not sure meaningful, enforceable consent can be obtained if it is required. I don't see how you run a website without constant A/B testing. I don't know that having the government get involved in the nitty gritty regulation of just how often website operators can change aspects of their sites or gather information from their site logs is socially useful - I will admit that my view is affected by living in China, where strong governmental control of internet sites is a given and in my opinion a powerful tool of oppression, and I don't think that controlling the great evil of A/B testing is the event that justifies the US going down the road of exerting such tight authoritarian control over website operators. I think that especially because enforcement of the Common Rule against A/B testing invites selective enforcement against sites whose activities create problems for the government.

[Ray Campbell]

I should say I read Professor Grimmelmann's addendum stating that "not all" A/B testing would require consent. He talks about a clear line and people on either side of it, but I have no idea where he thinks the line would be drawn - whether keeping results of manipulative experiments confidential within vast private enterprises puts you on the right side of the line, or whether they fall on the right or wrong side of a line based on whether they offend a certain refined sensibility, or whether they fall on the right or wrong side of the line based on how the government views the website. I think declaring there is a line but providing little guidance on where it lies is kind of a big problem.I think the line will prove much harder to draw than in the comforting examples he cites - doctors treating patients in their offices, historians reviewing recent events - because the day to day operation of websites involves testing of different experiences on randomly selected groups of people. I don't think that pervasive, randomized, experience shaping and measurement applies to the comforting "we know how to draw the lines" examples he cites.

[Steve Wilson (@Steve_Lockstep)]

Michelle, you're quite right of course that consent is not actually required for all ethical experimentation. But it was Facebook and the mood experiment authors who claimed they did have "informed consent", on the basis of a fragment of a sentence buried in the data usage agreement. If you're saying the experiment did not require consent, then the experiment designers themselves don't seem to understand that, because they tried to argue in favor of consent. And now the very same people are going on to steer a revised research policy. I'm afraid it's a sham.

[Ray Campbell]

The more I think about this the more interesting it is because it really points out the gap in ethics between scientific research and marketing. For scientific research that involves manipulation of people, we have a well developed body of ethical guidelines. For marketing, we don't, but I believe that the manipulation and testing engaged in by marketers is just as intrusive and at least as likely to affect people. It's not as if the goals of marketing are per se harmless - getting people to buy cigarettes or Big Macs or just stuff they can't afford can have a real negative impact. Marketers run randomized tests designed to determine the most effective ways to impact people's thoughts and moods, and engage in extremely sophisticated analysis to determine how much of the desired behavior was generated. In our society, marketing is pervasive, and impacts all of us in ways we don't even realize, conducted by people that (in my exposure to them) are pretty much amoral in the way they approach the task of developing, implementing, testing and tracking ways to manipulate people. I think it's worth thinking through why a published academic paper lives under one set of rules, but a combined TV-Internet ad campaign designed to impact moods and to get people to take specific actions lives under another regime entirely.

[Michelle Meyer]

Steve,

I'm very glad to see you acknowledge that not all experiments require, either as a matter of law or as a matter of ethics, explicit, detailed informed consent. I'm afraid that's something that has been lost in the conversation about the Facebook experiments and which threatens vital research in all sorts of domains (not just behavioral website testing).

Yes, there were various attempts to justify the PNAS experiments by pointing to the word "research" in the ToS which, it turns out, did not actually exist in the version of the ToS in operation at the time the experiments began. I'm not defending that. And frankly, the PNAS authors brought a lot of this on themselves (specifically, the alarmist, unscientific media frame, repeated by EPIC in its FTC petition, that "Facebook messed with people's emotions" just to satisfy its curiosity) by hyping the results of their experiments (you cannot conclude from the experimental design that the intervention had ANY effect on users' mood; and prior research suggested that Facebook's existing algorithms, which tend to prioritize and compactly present positive news feed items, "mess with" billions of users' moods by making them feel worse about their comparatively lackluster lives). And I'm not especially happy that in developing its internal guidelines, the one external person to whom Facebook appears to have reached out is the Cornell researcher without (to my knowledge) any research ethics training and with whom they collaborated on the very PNAS experiments that got them into this mess. Huh? That's not a good sign, if true. I'll be interested to see what data scientists from Facebook have to say about all this at the two upcoming conferences I linked to in the OP.

But ultimately I'm less interested in what justifications of the experiments were made and more interested in what, if anything, would in fact justify these and similar experiments. My Wired piece (http://www.wired.com/2014/06/everything-you-need-to-know-about-facebooks-manipulative-experiment/) lays out how, in my view, an IRB easily and consistent with federal -- and, not incidentally, Maryland state -- regulations could have approved these experiments more or less as-is, with the exception, perhaps, of debriefing after-the-fact. That theory rests on the regulatory exception for minimal risk research that couldn't be conducted with the usual robust informed consent. So when you see, in my co-authored Nature piece (http://www.nature.com/news/misjudgements-will-drive-social-trials-underground-1.15553?WT.mc_id=TWT_NatureNews), two theories defending the experiments -- one resting on a minimal risk exception to informed consent and one resting on there having been informed consent via the ToS -- you can infer which theory is mine.