Journals typically shy away from publishing data and text readers have seen before — but amidst the newly established norms of open science and data sharing, what counts as a prior publication?

We’re asking ourselves that question after learning that JAMA Pediatrics has rejected a letter rebutting a recent study in the journal about sexual assault on college campuses after deciding that posting the letter on PubPeer is a prior publication.

The submitted letter (which you can read here) was co-authored by independent consultant, therapist and researcher Jim Hopper, who is also a Teaching Associate in Psychology at Harvard Medical School. It concerned a 2015 paper published in JAMA Pediatrics, which suggested that the long-held belief that most rapists on college campuses are repeat offenders may be false. The findings can have major implications for university efforts to stop assaults, as institutions weigh whether to divert resources towards punishment (if serial offenders are largely responsible) or prevention (if most men only commit assaults once).

Hopper has questioned the validity of the authors’ findings, arguing that a problematic methodology clouds the true picture; the authors told us they stand by the paper’s data and conclusions.

Frederick Rivara from the University of Washington in Seattle, editor-in-chief of JAMA Pediatrics, told us:

By disseminating his letter on PubPeer post, this qualified as publication and we don’t publish things already published

That logic didn’t ring true for Hopper, who told us he was annoyed that the journal asked him to squeeze his vast criticisms into a 400-word document, only to receive a rejection in the end.

The phenomenon of rejecting papers that have already been published elsewhere — part of what’s known as the “Ingelfinger Rule” — was originally intended to discourage scientists from publicizing content that was not yet peer reviewed.

The study has received some news coverage, and has so far been cited four times, according to Thomson Reuters Web of Science.

After receiving some of the paper’s initial data from Kevin Swartout from Georgia State University (GSU) in Atlanta, the first author of the paper, Hopper hired an independent consultant, Allison Tracy, to test the paper’s findings.

When Tracy and Hopper discovered flaws in the study, and received what they considered to be insufficient clarifications from the journal and the study’s authors, Hopper informed the research integrity staff at GSU, and was told that they were going to carry out an investigation into the case. A GSU spokesperson declined to comment.

THE BACKSTORY

In recent years, a lot of attention from policymakers has been dedicated to issues around sexual assault on college campuses in the United States — inspired, in part, by the release of the film “The Hunting Ground.”

Many of the policies about preventing sexual assaults on campus have been influenced by the work of David Lisak — formerly Hopper’s graduate school mentor. One 2002 study has been particularly influential, Hopper noted — it showed that out of the 1882 men assessed for acts of sexual violence at the University of Massachusetts in Boston, 120 admitted to carrying out such acts. Out of the group who admitted to committing rape, around two thirds were repeat offenders, who, on average, confessed to committing 5.8 rapes. In total, over 90% of the rapes committed by the 120 rapists were perpetrated by the two thirds who were repeat offenders. The 2002 paper has been cited 92 times.

For years, said Hopper, federal regulators and politicians have relied on Lisak’s work when creating policies around sexual assault on college campuses. He added:

These points have huge implications for prevention and response because if someone comes to you and says they were raped, the odds are that the person who raped them has done it before and will do it again.

But some researchers (particularly those who work in prevention) are worried that the focus on Lisak’s two-thirds and 90% statistics have led policymakers and higher education administrators to conclude that resources should be devoted almost exclusively to identifying and stopping repeat offenders — that is, to law enforcement rather than prevention, said Hopper. He explained:

An argument some have made is that you can have the best prevention programs and roll them out on every campus in the country, but if a man is a repeat rapist, he’s not going to care about those prevention messages and only catching and expelling him or locking him up will solve the problem.

A “SERIAL” PROBLEM?

In the JAMA Pediatrics paper, Swartout and colleagues suggested that those concerns might be founded, since they suggest that most men only commit assault over one time period. The researchers assessed two samples with a total of 1642 participants, and found 172 men responsible for committing at least one rape from the age of 14 to the end of college. They report in their paper:

Most men (67 [72.8%]) who committed college rape only perpetrated rape during 1 academic year.

Hopper, however, criticized the JAMA Pediatrics paper for how its authors defined “serial rapist.” For example, they only counted completed rapes and excluded attempted rape, Hopper said. Another caveat, said Hopper: To avoid double-counting rapes reported on separate survey items — for example an item on rape of an intoxicated person and another item on rape involving anal penetration — the authors reduced their data down to binary rape variables for each assessment period. For each year, they gave each man a “1” for committing at least one rape or “0” for not. But this system can’t account for rapists committing multiple rapes within the same year of college, Hopper added — the only way for a rapist to be labelled as a “serial” offender in the Swartout et al study would be if they received a “1” in different years of college.

Based on his analysis, Hopper said the JAMA Pediatrics data show the opposite trend:

As I’ve shown on PubPeer for the publicly available dataset of theirs, in each year of college, the majority of guys who admitted to committing rape or attempted rape, admitted to doing it more than once — even within every individual year of college the majority were serial offenders who accounted for over 85% of all rapes and attempted rapes.

Tracy, who is based at Wellesley Centers for Women in Massachusetts, released a 75-page technical analysis of the paper’s methodology, and also published a seven-page executive summary of her findings. She told us:

I went into great detail trying to replicate and validate the findings that they report but the more I looked, the worse shape it appeared to be in. In a nutshell, I found technical problems with the data and the estimation of the model as well as inconsistencies in the reported results.

She added that

…even though both the code and the data were provided by Swartout himself, the programming code for creating the analysis dataset did not replicate the analysis dataset.

THE AUTHORS RESPOND

On behalf of all the paper’s authors, co-author Martie Thompson from Clemson University in South Carolina, sent us a statement:

Hopper, Lisak, and Tracy have raised concerns about the Swartout et al. (2015) article that challenges the serial rape perspective on college men who perpetrate persistently across their enrollment in higher education. The paper has been re-reviewed by JAMA statistical consultants and editors. At their direction, we have thoroughly reviewed all of the data analyses, as well as conducted an additional analysis that was requested. The outcome was an editorial decision that the paper is sound, requires no changes, reaches conclusions supported by the data, and that the implications stand as published. We look forward to continuing our studies, future work by other scientists, and collaborating to apply the findings to policy and practice.

That’s not good enough for Hopper, who has asked that the paper be retracted.

But a retraction was never on the table, Rivara said:

We never intended to issue a retraction; [Hopper] thought we should but I never did.

The paper has, however, received a correction — in December 2015, the journal added a note to “Trajectory Analysis of the Campus Serial Rapist Assumption.” It reads:

In the Original Investigation titled “Trajectory Analysis of the Campus Serial Rapist Assumption,” published online July 13, 2015, in JAMA Pediatrics, there were inconsistencies in missing data between the data used for the published analyses and the publicly available derivation data, which affected 2 cases. After correcting for these errors, most of the frequencies and statistics reported in the Results section differ slightly. All interpretations and conclusions remain the same after correcting for these errors. A corrected article with corrections to the Abstract, text, Tables, and Figure has been published online. In addition, this article was previously corrected on August 13, 2015, to fix a column heading in Table 3.

Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.

Share this: Email

Facebook

Twitter

