Do you work in academic research? If so, you probably have a view on peer review. The system is at the heart of scholarly communication – and it elicits strong opinions from across the community. Many have concerns about the integrity of the process – as demonstrated by the popular hashtag #sixwordpeerreview, which mocks short, unhelpful feedback.

Our year-long research project set out to explore the best approaches to peer review, canvassing the opinions of academic authors, reviewers and the journal editors who oversee the process.

Researchers from across the sciences, social sciences, medicine and humanities were asked to complete a survey or take part in focus groups in China, the UK and South Africa. More than 7,400 responded, answering questions on the purpose of peer review, the prevalence of ethical issues, timeframes, and how comfortable (or not) they were with the different peer review models.

Why examine peer review?

Peer review is essential element of getting research published, yet a significant number of people feel that the system is broken. We set out to explore the expectations and the reality of the process. “The worst reviews are short, snitty, patronising and not remotely useful. The best are critically engaged, add something and improve the quality,” we were told.

What should peer review achieve?

It should improve the quality and authenticity of a paper and check for an appropriate and robust methodology.

We found the expectations around detecting plagiarism were higher among science, technology and medical (STM) researchers than those in the humanities and social sciences. Yet STM researchers said the biggest real-world failing of peer review was in spotting academic fraud. But there was, at least, one area where reality exceeded expectation across the board: correcting spelling, grammar and punctuation.

What about ethics?

Ethics in peer review has seen a huge amount of coverage (in relation to gender bias, for example), yet we experienced a marked drop-off in responses to this part the survey (11% chose not to answer these questions). There was also less unprompted discussion about ethics in the focus groups.

Those that did answer said that regional and seniority bias were the most common issues, and identity and gender bias were the least. Double-blind peer review (where the reviewers’ and authors’ identities aren’t known to each other) was widely viewed as the best way to prevent discrimination.

For example, one academic told us: “I used to be at a university which is low ranking in my current field. When I was there, I couldn’t get a paper accepted but now I am at a well-respected institution, I feel some papers are accepted too easily.”

But who should manage ethical issues? Researchers couldn’t quite agree. Some 44% of humanities and social science researchers said the responsibilty should rest with the journal editor and editorial board, and 42% of STM researchers said it should be split equally between the journal editor and publisher.

Researchers consider double-blind peer review to be the model they are most comfortable with

How long should a review take?

Most reviewers considered up to 60 days to be a reasonable turnaround time for initial feedback, and almost all said that they had achieved this on the most recent article they reviewed. But many authors reported waiting longer.

The responses from journal editors highlighted the fact that the actual review was just one element of the process – they were also likely to be be checking for plagiarism, finding and selecting appropriate reviewers and going through a tiered editorial decision-making process, much of which was invisible to authors.



Everyone wants the “right” reviewer for their paper, of course, but our results show that many aren’t aware of how long it takes to identify them and get them to agree to write and deliver a review on time.

Which model is best?

We asked researchers to rate the level of comfort they would feel with different models of peer review (single-blind, double-blind, open, open and published, and post-publication) being applied to their work. Double-blind came out top across all disciplines for authors, reviewers and editors.

Respondents were decidedly polarised about newer models of peer review such as open review, open and published, and post-publication review. Some were more comfortable than others. For example, one researcher in the biological sciences said that a benefit of open review was that the authors knew whether the reviewers were their competitors. But responses ranged from very positive to deeply uneasy.

So although there are regular debates about the pros and cons of peer review and how it could and should look in future, our research found a fairly conservative view from researchers. There seems to be a preference towards tweaks to the current system rather than a radically new way of assessing research.

• Elaine Devine works for the Taylor and Francis Group, which published the white paper Peer Review in 2015: a global view .

Join the higher education network for more comment, analysis and job opportunities, direct to your inbox. Follow us on Twitter @gdnhighered. Email article pitches to us at highereducationnetwork@guardian.co.uk

