Open Science Collaboration/CC BY 4.0

A data-sharing campaign that involves awarding colourful badges is a surprisingly powerful incentive to get scientists to publicly share the data behind their research, according to the results of a 17-month experiment.

In 2014, the journal Psychological Science announced that it would award badges — in the form of blue or orange icons at the top of a research paper — to papers that made relevant data or research materials publicly available. Eric Eich, who was editor-in-chief of the journal at the time, got the idea from a suggestion by social psychologist Brian Nosek about three years ago. Nosek was in the midst of setting up the Center for Open Science (CoS), a non-profit organization in Charlottesville, Virginia, that is dedicated to increasing the openness and reproducibility of research.

“There is a sense of accountability when someone applies for a badge”

Working independently of staff at Psychological Science, researchers at CoS checked each article’s claims about data availability. They also checked data availability for articles in four other psychology journals. Their results are published in PLoS Biology1.

By the first half of 2015, the proportion of articles in Psychological Science stating that data were available had shot up to around 40%, even as rates in four other psychology journals remained at below 10%.

Out of 171 eligible papers published between July 2014 and May 2015, 43 received a badge for open data — although the CoS researchers found that not all of these actually made all relevant data available in a way that other researchers could readily use. But badges were a better indicator of data availability than were mere statements that resources could be obtained, says Mallory Kidwell, a CoS project coordinator who led efforts to analyse nearly 2,500 articles. “There is a sense of accountability when someone applies for a badge,” she says. Rates also rose for Psychological Science research papers that declared the availability of research materials, but these patterns were less pronounced.

Multiple influences

It is impossible to prove that the badges themselves increased sharing, concedes Nosek. Psychological Science asked authors of accepted papers whether they wanted to earn a badge by making their research resources available, so the mere act of asking might have had an influence. So might advocacy by Eich. But Nosek thinks that the combination is important — a journal stating that it values the sharing of data and materials, evaluating whether scientists have done so, and then publicly signalling papers that conform.

Besides Psychological Science, nine other journals have now implemented badges or have announced plans to do so. Eich, who was not involved in the analysis, thinks that the results may help to convince editors at other journals to give badges a try. “They are bound to hear scattered complaints about badges being gimmicks or contrivances, but the empirical evidence presented in Kidwell et al.'s new paper should allay most serious concerns,” he says.

Information scientist Youngseek Kim at the University of Kentucky in Lexington calls badges a creative way to increase data sharing. The choice to share data is heavily influenced by expected norms of behaviour — and badges serve to increase that expectation, he says.