Most of the attention has been focused on the particulars: how almost 700,000 Facebook users were subjected to a psychological experiment without their knowledge or explicit consent, the decision to manipulate their News Feeds to suppress either positive or negative updates, and why this study was accepted to a journal without academic ethics approval. But beyond this horizon, some truly difficult questions lie in wait: What kinds of accountability should apply to experiments on humans participating on social platforms? Apart from issues of consent and possible harm, what are the power dynamics at work? And whose interests are being served by these studies?

But this is not the first time emerging technologies have come into conflict with the reigning beliefs about how human experiments should be done. Not unlike today, the late 1950s and early 1960s were a time of rapid change for social science, with new approaches and a growing appetite for experimental interventions. “Manipulative experimentation,” according to Shils, “is not a relation between equals; it is a relationship in which power is exercised.” For him, the less a subject is informed about or agrees with the aims of the experimenter, and the less intelligible the means of the study, the more ethically problematic it becomes.

We have now had a glimpse within the black box of Facebook’s experiments, and we’ve seen how highly centralized power can be exercised. It is clear that no one in the emotional contagion study knew they were participants, and even now, the full technical means and mechanisms of the study are only legible to the researchers. Nor can we know if anyone was harmed by the negatively skewed feeds. What we do know is that Facebook, like many social media platforms, is an experiment engine: a machine for making A/B tests and algorithmic adjustments, fueled by our every keystroke. This has been used as a justification for this study, and all studies like it: Why object to this when you are always being messed with? If there is no ‘natural’ News Feed, or search result or trending topic, what difference does it make if you experience A or B?

The difference, for Shils and others, comes down to power, deception and autonomy. Academics and medical researchers have spent decades addressing these issues through ethical codes of conduct and review boards, which were created to respond to damaging and inhumane experiments, from the Tuskegee syphilis experiment to Milgram’s electric shocks. These review boards act as checks on the validity and possible harms of a study, with varying degrees of effectiveness, and they seek to establish traditions of ethical research. But what about when platforms are conducting experiments outside of an academic context, in the course of everyday business? How do you develop ethical practices for perpetual experiment engines?