In psychology, there are a few studies so famous and influential that they have proper names: The Good Samaritan Study, the Asch Obedience Study, the Marshmallow test, etc, etc.

Approaching this echelon is the “Cookie Monster Study”, an increasingly-famous study of social power. If you don’t already know it, here’s a quick summary:

Ward and Keltner (1998) examined whether power would produce socially inappropriate styles of eating. In same-sex groups of 3 individuals, 1 randomly chosen individual (the high-power person) was given the role of assigning experi- mental points to the other 2 on the basis of their contributions to written policy recommendations concerning contentious social is- sues. After group members discussed a long and rather tedious list of social issues for 30 min, the experimenter arrived with a plate of five cookies. This procedure allowed each participant to take one cookie and provided an opportunity for at least 1 participant to comfortably take a second cookie, thus leaving one cookie on the plate. Consistent with the prediction, high-power individuals were more likely to take a second cookie (see Figure 6). Coding of the videotaped interactions also revealed that high-power individuals were more likely to chew with their mouths open and to get crumbs on their faces and on the table. Male participants ate in more disinhibited ways as well, lending further support to our power-based hypothesis, to the extent that gender is equated with power.​(Keltner, Gruenfeld, & Anderson, 2003)​ (p. 277)

This study has become influential in the public sphere. It has been covered extensively in the national news media for over a decade (here and here and here and here and of course by NPR here and here and here…and, well, lots more places). There’s a cute YouTube video explaining the study which has been viewed over 100,000 times. It’s been lifted up into life advice, via a commencement address that went viral by Moneyball author Michael Lewis (“Don’t Eat Fortune’s Cookie“, delivered at Princeton in 2012). Finally, one of the authors of the study, Dacher Keltner, has written and discussed the study extensively, penning both thought-provoking essays and a pop-science book which feature the Cookie-Monster Study prominently.

It’s easy to appreciate the appeal of this study–it neatly encapsulates and supports our worry that power changes us, shaping us into less moral and sensitive creatures. As Shankar Vedantam breathlessly put it during a recent NPR interview with Keltner:

So it’s fascinating because, of course, what you see in these lab experiments is often reflected on much, much bigger stages, where you see people in power abusing that power – you know, having affairs, cheating and, you know, falsifying financial returns. And, you know, at one level, the conventional view, I think, is sort of to say, these are just people who were bad people who rose to the top. But what you’re suggesting is actually something more complex and, in some ways, much sadder, which is that these might not be bad people who rose to the top, but these might be good people who rose to the top, and power has made them bad. — Shankar Vedantam, NPR

Given the intrinsic interest in the Cookie Monster Study, it is surprising to find that it hasn’t been extremely influential in the academic world: Google scholar says it’s only been cited 26 times. Why is that? Because the original study is an unpublished manuscript… so there’s nothing to cite. The study was introduced to the world via a 2003 review paper (quoted above) by Keltner and colleagues ​(Keltner et al., 2003)​. The review summarized the study and provided a summary figure, but cited only an unpublished manuscript:

Ward, G., & Keltner, D. (1998). Power and the consumption of resources. Unpublished manuscript.

Ok, but as the study became more famous surely journals would have been lining up to publish it. Why is it still unpublished? Well, according to Keltner when he moved from the University of Wisconsin to UC Berkeley he lost contact with the lead author (Ward), and with him all access to the data, methods, and materials, leaving him unsure about even the original sample sizes. A screen-shot of his email to me (Bob) about this is below. According to Wikipedia, Keltner moved to UC in 1996, so the data has long been gone and unavailable.

That’s strange, right? That means the Cookie-Monster Study isn’t an influential paper..it’s an influential memory of a study, which 20-years after the fact is still providing subject-matter for books, essays, and breathless popular-press coverage.

And then it gets even more strange. The review paper by Keltner and colleagues that introduced the Cookie-Monster study to the world has this figure summarizing the results:

​(Keltner et al., 2003)​ p. 277

Look carefully… notice that men ate slightly fewer cookies in the high-power condition? If there is an effect, it would have been for women only. So the figure doesn’t match the summary given in the review paper nor with the interviews and summaries given since then by Keltner. Apparently none of the review paper authors noticed this, nor the reviewers. The review itself has been cited over 2700 times, and I can’t see anyone mentioning the discrepancy. When I pointed out the error to Keltner, he still didn’t seem to notice, stating that the effect was “more pronounced in women” but “observed for both men and women”. But that’s not the in the least what the published figure shows. So not only is the Cookie-Monster study based on a memory, it is likely based on a faulty memory. There are other possible indications of this. For example, Keltner seems to be inconsistent about if there were 4 or 5 cookies served to the participants. In the YouTube video about the study he specifically mentions 4 cookies, whereas in other venues he insists that pilot testing showed 5 cookies were needed to license the powerful person to take another cookie. Maybe this is just a detail lost to editing in these pieces, but the fact remains that there’s no way left to check which account is correct.

None of this means that the Cookie Monster study is wrong–just that the sum of its influence is based on one researcher’s potentially faulty memory of what one of his students did 20+ years ago. The evidentiary value, at this point, is probably nil. It might not be the most suitable study to lift up for public discussion. When it is discussed, it would probably be best to make it clear that it is a recollection of a data long-since lost.

Interestingly, the effect could potentially be reliable. A 2008 study attempted to replicate Ward and Keltner. They used dyads with a confederate, so there are some concerns about demand effects. Yet they did find that those assigned to a high-power situation ate more cookies. ​(Smith, Jost, & Vijay, 2008)​. It’d be great to do an RRR on this.

Psychological Review, 265–284. doi: Keltner, D., Gruenfeld, D. H., & Anderson, C. (2003). Power, approach, and inhibition., 265–284. doi: 10.1037/0033-295x.110.2.265 Social Justice Research, 358–376. doi: Smith, P. K., Jost, J. T., & Vijay, R. (2008). Legitimacy Crisis? Behavioral Approach and Inhibition When Power Differences are Left Unexplained., 358–376. doi: 10.1007/s11211-008-0077-9