Drew Altizer Photography/CC BY 2.0

Academia.edu, a social-networking site for researchers, has announced an effort to get scores of academics to talk about research manuscripts together online before submitting them to journals.

Some users on academia.edu have been trialling out a feature called ‘Sessions’, which allows them to discuss a manuscript for 20 days in conversations that are neither fully public nor permanently available online. On 25 September, that feature will be rolled out to everyone on the site, says its chief executive officer and founder, Richard Price.

Price tells Nature how 'Sessions' works — and why it could encourage mass online conversations about research papers when other attempts have failed. This interview has been edited for length and clarity.

What exactly is a 'Session'?

The goal of Sessions is to solicit feedback from Academia’s network of 25 million users. If you're a site user, you will see new Sessions through your news feed of social activity. You can see Sessions created or joined by people you follow, and in topics you follow. Joining them means you’re in the room — you’re able to access the comments. Sessions are not marketed in any way. It’s purely viral.

We’re just coming out of a closed-stage trial, which has been wildly popular. Literally hundreds of people are coming together to discuss a single paper. On any given day right now, there are 6,000 Sessions under way.

The way it works is that on the left side of the page, you have the paper itself. On the right side, you see comments from people. Some of these are annotations. You can see what part of the paper an annotation relates to. It is quite common for the comments to be fairly hard hitting — people are generally critical. After 20 days, comments are archived and accessible to anyone who was part of the Session, but no new person can join.

What will draw people to participate in an online feedback forum?

Most of the time, it takes ages to get papers peer-reviewed. There are substantial delays. The basic way that peer review works is someone sends you an article to review, and it’s a low priority thing. You just get to it eventually. It can be months. Or, in rare cases, years. But people go to conferences, and spend tons of money and tons of time on getting there, and they just sit there and listen to papers and peer review them — critiquing them in Q&A sessions, which is basically a form of peer review. So, what’s the difference?

We feel the difference is this — it’s a group activity in the conference; you know who everyone else is. It makes intellectual discussion fun in a way that peer review isn't.

Haven’t people already tried creating online communities to critique papers?

There have been a lot of experiments on online peer review that have not worked. The question is: why haven’t they worked? There were basically two problematic issues. One was this concern about your comment being public on the internet, and for decades to come. The second issue was that most of the online-commenting experiments didn’t allow annotation — they were just like, 'Here’s the pdf', and there was a comment box at the end.

We thought a lot about how not to make those mistakes again. So, to design this product, we interviewed hundreds of academics in every field about why they go to conferences — to present papers, or to attend? And if they do attend, when do they ask a question — and why?

We decided that privacy is absolutely key. Comments stay in the room — they’re not recorded. These comments are not 'Google-able'. If a random bot hits this link, it doesn’t get in. It’s not watertight security. But then, neither is a conference. We want conversation — the kind of conversation you get over e-mail; the kind you get at a conference.

Why is Sessions different than just circulating a draft using e-mail?

Basically, my view is that the more feedback, the better. Academics have loads of different channels to get feedback, including conferences, asking their colleagues and now our Sessions tool. They all have their advantages.

The advantage of Sessions is you get to use this massive network. You would never actually e-mail 80 people; you might e-mail two or three. You would, however, go to a conference at which your audience might be as large as 80. To get access to this size of audience while being able to stay in your lab is unique.

Is the ultimate aim to improve papers before submitting to journals, or to replace peer review?

Our motivating question was: how do our users get feedback directly from their peers? You could argue that Sessions is a peer-review experience: all the people participating are academic peers in a room, but it’s not mediated by an editor or a journal. It’s mediated by a network, and people are just joining because they see the content.

I do think that it’s important to be innovating with peer review. And the line between feedback and peer review is very blurry. We’ll see how it goes.