Booklet helps public understand peer review

New US version aims to help people make sense of science debates in the news

By Julia Wilson Posted on 8 February 2013

The Author

Julia Wilson is Development Manager at Sense About Science, a UK charity that helps people make sense of science and evidence. She is responsible for the organization's fundraising activities and international work. Wilson previously coordinated their Voice of Young Science program, which encourages early-career researchers to take an active role in public debates about science. She has a degree in biology from the University of Manchester.

[divider] Every day brings new science claims into the public discussion - from news media, policy statements, advertising, websites, campaigns and social commentary. Making sense of it can be difficult, whether you're a parent concerned about the dangers of Wi-Fi in schools or a policymaker trying to understand claims about nuclear energy. Which claims should be taken seriously? Which stories are "scares" or hype?

A vital first step is understanding peer review: the process of critical scrutiny used by scientists to decide which research results should be published in a scientific journal. One of the first questions people can ask when trying to make sense of science stories is, "Is it peer reviewed?"

Over the past decade, Sense About Science has been working to promote an understanding of peer review in the UK among policy makers, journalists, social influencers and civic organizations.

Today we are launching a US version of our public guide to peer review: I Don't Know What to Believe.

The guide explains how research findings are reviewed for validity, significance and originality. It also gives a brief summary of how editors select reviewers and discusses the importance of ensuring that papers refer to previous work and provides information so that others in the field can see how the research was conducted.

We just made the announcement at the annual meeting of the Association of American Publishers Professional and Scholarly Publishing (PSP) Division in Washington, DC.

Understanding that published research has been scrutinized by other experts in the field is a valuable insight for the public and a useful first step in helping people distinguish between science, pseudoscience and opinion. And it illustrates that the scientific knowledge we regard as fact – such as the Earth revolving around the sun – is actually the result of many years of academic argument and gathering of evidence. It can help the public consider new research critically and to consider its evidence base, and not just believe new theories because they appear to "make sense."

Contrary to the fears of some researchers in the UK, the public seems quite able to understand that peer review is an indicator of scrutiny rather than the final word. Perhaps that should not be surprising. When we buy a washing machine, it has a kite mark. We know this means that it has passed some safety and operating standards. We also know that the washing machine won't work forever and, in fact, the kite mark doesn't guarantee it won't break down the day we get it home. But we know that the kite mark is important all the same.

Peer review seems to be a well-kept secret of the research community. When asked about peer review, most scientists' minds spring either to the paper they have in the system, or to the papers sitting threateningly in their inbox awaiting a reviewer's eye. When we first spoke to scientists about sharing peer review with the public, they didn't think anyone would be interested. Through workshops and discussions, we've encouraged scientists to stand back from the system and think of the role that peer review plays in wider society. [note color="#f1f9fc" position="right" width=400 margin=10 align="alignright"]

Sponsors

This guide has been produced and distributed with financial assistance and expertise from the following organizations:

We launched the UK version of the guideI Don't Know What to Believein 2005 and have since received over half a million requests for it. The requests reveal the broad range of people involved in passing on scientific information to the public: health-care workers, librarians, societies, public health officials, policymakers, technology companies, safety bodies, popular writers, educators, parenting groups and local government. These are the organizations that, when the newspapers move on from a scare story, are left picking up calls from anxious patients and their relatives.



We have also seen a change in the way the UK media reports on the status and quality of research. Last year, the BBC made it a policy to always reference where research comes from in their articles, and most of the national newspapers are doing this now. This starts the public conversation about where science claims are coming from and gives people the opportunity to investigate further and weigh up conflicting stories.

US version and peer review campaign

Now, we're taking I Don't Know what to Believeto the US. The public guide explains the peer review process and the importance of the status of evidence when weighing up conflicting claims about science. Working with publishers, learned societies, patient groups and journalists, we updated the content on issues of research misconduct and the different models of journal publishing and peer review. We also include a few examples of issues in the United States.



We hope to create public interest in the value of peer review as part of an annual "Peer review matters" campaign in the US, which will include talks for teachers, heath-care workers and members of Congress, and events at science festivals and in the research community. We will also promote our "I don't know what to believe" e-button, for organizations that publish peer-reviewed scientific information on their websites. Patients searching online for information about medical treatments might come across many different websites and claims. By clicking on the e-button, they will get a copy of the peer review guide, which will help them better assess the information they find.

"We have to establish an understanding that the status of research findings is as important as the findings themselves," said Tracey Brown, Managing Director of Sense About Science. "This understanding has the capacity to improve the decisions we make across all of society."

Asking "Is it peer reviewed?" will be a starting point to help people cut through the hype and make sense of some of the current debates that are making the headlines in the US on issues such as genetic modification, chemicals and childhood immunizations. We want to encourage people to consider the evidence and question where claims are coming from. By giving people tools to ask questions about the status of research, we can help them transcend the noise of the debate to focus on the science. [note color="#f1f9fc" position="center" width=800 margin=10]

The Kite Mark

Just as a washing machine in the UK can have quality "kite mark," peer review is a kind of quality mark for science. It tells you that the research has been conducted and presented to a standard that other scientists in the field accept. At the same time, peer review does not claim that the research is perfect – just as a kite mark does not guarantee that the washing machine will never break down.[/note]

What is Sense About Science?

Sense About Science (SAS) is a UK charitable trust that equips people to make sense of science and evidence on issues that matter to society. With a network of more than 5,000 scientists, the organization works with scientific bodies, research publishers, policymakers, the public and the media to lead public discussions about science and evidence. Through award-winning public campaigns, it shares the tools of scientific thinking and the peer-review process. Sense About Science's growing Voice of Young Science network engages hundreds of early career researchers in public debates about science.

Related Links