Photo: Sundance Institute

Update: A response from Kirby Dick, the writer and director of the film, has been appended to this article.

The Hunting Ground, a documentary that aired on CNN Sunday night after garnering a great deal of publicity on the festival and campus circuits, tells some horrific stories about sexual assaults on college campuses, and in doing so it raises a number of important, damning questions about how campus administrators handle rape allegations. Unfortunately, it also presents some extremely questionable research about campus rape in far too credulous a light.

(Updated with a statement from Kirby Dick, the film’s director, at the bottom.)

It’s probably important to get our controversies straight here. One set of criticisms leveled against The Hunting Ground has to do with its portrayal of an alleged rape involving Harvard Law School students. Both Emily Yoffe of Slate and a group of 19 HLS professors have argued that the film misrepresents the facts of that incident in important ways.

The social-science issue is separate, and it involves Dr. David Lisak, a retired clinical psychologist who has greatly influenced current anti-campus-rape efforts — his research has even been cited in materials published by the White House. Twenty-five minutes into the film, he lays out his general theory of the campus-rape problem: “The vast majority of men don’t rape, won’t rape, haven’t raped,” he says. “So when you start looking at, then, the rapists who are committing these crimes, it is the repeated offenders who are the core of this problem.” Text appears onscreen: “Less than 8% of men in college commit more than 90% of sexual assaults.” Additional text below cites the source for these numbers: a 2002 paper by Lisak and Dr. Paul M. Miller.

This claim marks an important tone-setting moment in The Hunting Ground, and it’s important to understand its ramifications. If Lisak, who is presented in the film as an authority on the subject of campus rape, is correct and the “core” of the campus-rape problem is serial offenders, then addressing campus rape mostly means targeting a very small slice of hardened predators who are committing an astounding proportion of the assaults that occur on a given campus. Obviously, attacking a rape problem like this would be different from attacking a rape problem in which assaults are distributed across a wider set of perpetrators.

But Lisak and Miller’s 2002 paper, published in Violence and Victims, is the only research providing empirical support for such an extreme version of this serial-rapist theory. And to call that paper “controversial” doesn’t capture the half of it.

Here’s the short version: Miller and Lisak’s data are drawn from surveys conducted of 1,882 men at UMass-Boston, a commuter school with no campus housing in which the men were asked if they had ever raped someone or attempted to rape someone. This was not a group of traditional undergraduates — the average age of those surveyed was 26.5, with the range running from 18 all the way to 71. “More than 20% were over age 30, and nearly 8% were over 40,” the authors noted. Most of these men probably weren’t full-time students, since UMass-Boston is largely oriented toward part-timers. Moreover, as Robby Soave of Reason reported in July, it’s not even clear that all of the men surveyed were even students at all. Lisak and Miller didn’t collect their own data, but rather adapted data that had been previously collected by other researchers for other purposes over a number of years prior to the publication of their paper. These researchers had handed out surveys at “main pedestrian traffic points on campus” at UMass-Boston (respondents who filled out a survey got a few bucks for their efforts) but, according to Soave’s reporting, never checked whether the respondents were actually students, because for the purposes of the data they were collecting it didn’t matter.

Setting aside the fact that it’s unclear how many non-students were surveyed, it’s also hard, from the point of view of understanding campus rapists, to even interpret the rapes that were reported, given UMass-Boston’s lack of campus housing and campus social life like the sort found at more traditional schools. If a 29-year-old part-time commuter student says that, yes, in his life he has raped someone, that obviously doesn’t make the rape itself any less egregious, but it’s difficult to assess whether the rape can be seen as connected to his life as a college student at all. It would be a stretch, in other words, to describe him as a “college” or “campus” rapist without knowing more. The authors themselves certainly seemed to understand that: “There is not a single statement in the paper about assaults taking place on or near a campus; there is not a single reference to the campus environment,” wrote Linda M. LeFauve, who has also raised important questions about Lisak’s research in Reason.

In short, Lisak and Miller’s study has very little to do with the conversation about sexual assault The Hunting Ground is concerned with. The respondents were older, differed in meaningful ways from “traditional” college students, and may well have been mostly reporting rapes that had nothing to do with college anyway. From a research perspective, it’s often dangerous to extrapolate the results of a single survey to a broader population; in this case, to say the UMass-Boston numbers can be applied to college students on the whole is quite irresponsible, especially in light of the lack of other published findings that back up Lisak’s serial-predator theory. “I am not familiar with any research that corroborates or replicates David Lisak’s findings,” Christopher Krebs, a well-known sexual-assault researcher at RTI international, said in an email.

And yet this baseless statistic caught on, spread like wildfire, and has been cited not just by the White House and the producers of The Hunting Ground, but by countless other activists and academics as well. As Yoffe has pointed out, Lisak has contributed to the misunderstanding by propagating the notion that his research can be extrapolated to the broader college population, despite the fact that he and Miller explicitly cautioned that their research “cannot be interpreted as estimates of the prevalence of sexual and other acts of violence” on a campus or anywhere else because of the lack of a random sampling procedure. (Lisak downplayed this caveat to Yoffe as “a standard disclaimer for any study.”)

In addition to the numerous problems with the Lisak-Miller study, a more recent, rigorous attempt to understand campus rapists turned up very different results anyway. In July, JAMA Pediatric published a paper by a group led by Dr. Kevin M. Swartout that highlighted the popularity of the serial-perpetrator theory and the thinness of evidence behind it (“Although the serial rapist assumption is widely taken as fact by politicians and the popular press, it appears to be premised on a single source”), and then proceeded to use the “2 largest existing longitudinal samples of college men’s sexual violence,” each sample drawn from students at one university (both in the same region of the country) and adding up to 1,645 respondents total, to test that theory.

Swartout and his colleagues looked at their data in a slightly different way than Lisak and Miller did; they found that of the men in their sample who committed at least one completed act of penetrative rape (about 11 percent of the sample), 75 percent only did so during one academic year. “Although a small group of men perpetrated rape across multiple college years, they constituted a significant minority of those who committed college rape and did not compose the group at highest risk of perpetrating rape when entering college,” the authors write. Therefore, their data suggest that “at least 4 out of 5 men on campus who have committed rape will be missed by focusing solely” on serial offenders of the sort highlighted by Lisak and Miller. No one survey can tell the whole story, of course, but these numbers strongly suggest a more sophisticated, nuanced approach is needed.

This gets to the heart of why Lisak’s viral zombie statistic is so damaging: It simply leads people astray. No one is arguing that serial sexual predators don’t exist in college settings. But there’s startlingly little evidence for Lisak’s claims that they commit the vast majority of rapes on campus. Rather, it appears that for whatever reason — and this is where further research is so crucial — many men in college are capable of committing rape in a “limited” (for lack of a less terrible term) manner. And yet for years, Lisak and Miller’s paper has served as a flashing neon light pointing researchers and activists in what is very likely the wrong direction. This could go down as a major scientific misstep.

The Hunting Ground’s producers aren’t the first people to have become morbidly enamored with David Lisak’s serial-rapist theory, but it’s unfortunate, given the reach and influence of their film, that they did.

Update: Kirby Dick sent Science of Us the following statement: “Every statistic in our film is accurate, including the statistics on repeat offenders, and nothing in this article proves otherwise. Moreover, no individual or institution has asked us to retract a single fact error in the film. Singal’s headline “The Hunting Ground Uses a Striking Statistic About Campus Rape That’s Almost Certainly False” is itself false. The statistic the headline refers to is that ‘8% of college men commit more than 90% of sexual assaults.’ That statistic is completely consistent with the data from the two studies conducted on college repeat offenders (Lisak and Swartout). Singal could have easily confirmed this if he had reached out to me for comment.”