IMAGE: STACEY PENTLAND PHOTOGRAPHY

Reproducibility, rigor, transparency, and independent verification are cornerstones of the scientific method. Of course, just because a result is reproducible does not necessarily make it right, and just because it is not reproducible does not necessarily make it wrong. A transparent and rigorous approach, however, can almost always shine a light on issues of reproducibility. This light ensures that science moves forward, through independent verifications as well as the course corrections that come from refutations and the objective examination of the resulting data.

“…scientific journals are standing together in their conviction that reproducibility and transparency are important…” IMAGE: SORBETTO/ISTOCKPHOTO.COM

It was with the goal of strengthening such approaches in the biomedical sciences that a group of editors representing over 30 major journals, representatives from funding agencies, and scientific leaders assembled at the AAAS headquarters in June of 2014 to discuss principles and guidelines for preclinical biomedical research. The gathering was convened by the U.S. National Institutes of Health, Nature,* and Science.

The discussion ranged from what journals were already doing to address reproducibility and the effectiveness of those measures, to the magnitude of the problem and the cost of solutions. The attendees agreed on a common set of Principles and Guidelines in Reporting Preclinical Research (www.nih.gov/about/reporting-preclinical-research.htm) that list proposed journal policies and author reporting requirements to promote transparency and reproducibility.

The new guidelines suggest that journals include in their information for authors their policies for statistical analysis and how they review the statistical accuracy of work under consideration. Any imposed page limits should not discourage reproducibility. The guidelines encourage using a checklist to ensure the reporting of important experimental parameters, such as standards used, number and type of replicates, statistics, method of randomization, whether experimenters were blind to the conduct of the experiment, how the sample size was determined, and what criteria were used to include or exclude any data. Journals should recommend the deposition of data in public repositories where available and link data bidirectionally to the published paper. Journals should strongly encourage, as appropriate, that all materials used in the experiment be shared with those who wish to replicate the experiment. Once a journal publishes a paper, it assumes the obligation to consider publication of a refutation of that paper, subject to its usual standards of quality.

The more open-ended portion of the guidelines suggests that journals establish best practices for image-based data (such as screening for manipulation and storing full-resolution archival versions) and how to describe experiments more completely. An example for animal experiments is reporting the source, species, strain, sex, age, husbandry, inbred and strain characteristics, or transgenic animals, etc. For cell lines, one might report the source, authentication, and mycoplasma contamination status. The existence of these guidelines does not obviate the need for replication or independent verification of research results, but should make it easier to perform such replication.

Some of the journals at the meeting already had implemented all or most of these principles and guidelines. But the important point is that a large number of scientific journals are standing together in their conviction that reproducibility and transparency are important issues.† As partners to the research enterprise in the communication and dissemination of research results, journals want to do their part to raise the standards for the benefit of all scientists and the benefit of society. The hope is that that these guidelines will not be viewed as onerous, but as part of the quality control that justifies the public trust in science.