Popular Science editors recently shut down the site’s comment section for most articles because they feel uncivil comments reduce the credibility of the science presented in the articles themselves. It’s a step, the editors hope, in helping roll back the “war on expertise.”



But some immediately questioned if it was also a rollback of the open ethos of the Web and a rejection of the 21st century media commitment to two-way conversation. The science community has been abuzz, and the wider media world is also chiming-in.

Simultaneously, a group of scholars convening for the Sackler Colloquium at the National Academy of Sciences contemplated these very questions about distortion of science in a digital media era. Few definitive policy-oriented conclusions there, but lots of plans for future research agendas.

The two events highlight a gnawing problem unsettled at any level of the information and communications system.

The core issues for the media are not new. How much should news outlets let the public “in”? How prominent should those voices be on a website? What editorial strategies and technologies can promote moderation and civility? Jeff Sonderman and Andrew Beaujon at the Poynter Institute have made this perpetual debate something of a beat, in and of itself.

Internet “trolls” have long known that posting early and often can tilt discourse toward their viewpoint and scare away more reasonable commenters. But well-intentioned efforts to moderate user comments in some cases have resulted in dubious or unintended outcomes.

Questions for News Editors … and Thomas Jefferson

Those on the digital side of a newsroom are used to struggling with questions of trolls and flaming, “sock-puppeting,” and legal questions about hosting hate speech. Many have looked to solutions such as comment registration through Facebook — which requires real names, thus limiting anonymous messages — or deploying tools like Disqus, which allows votes up or down on individual comments, relying on, and hoping for, the wisdom of the “crowd.”

Every business-side news manager must ponder, at some level, a ruthlessly practical cost-benefit question and calculus: Does all this “engagement” entice sufficient Web traffic that can be monetized through “CPMs” — advertising rates per 1,000 views? Or does messy hoi polloi engagement instead hurt the product brand, diminishing subscription rates? Are comment forums — civil and uncivil alike — a service that readers appreciate and that make a news product more valuable, with some journalists now seeing their role as that of conversation “convener”? Meanwhile, journalists love to see their work light up with hundreds of comments, but the tradeoffs can involve tolerating incivility and enduring personal attacks — as has long been the case when reporting on climate change.

Dominique Brossard addresses ‘the nasty effect’ of online incivility.

There are loftier issues at stake, too, such as those involving cultivating an informed public and serving democracy. Popular Science raises exactly these questions in its editorial policy announcement; its rationale leans heavily on research publicized earlier this year in a New York Times opinion piece by the University of Wisconsin science communications scholars Dominique Brossard and Dietram Scheufele. That article was based on a study they co-authored in the Journal of Computer-Mediated Communication, “The ‘Nasty Effect’: Online Incivility and Risk Perceptions of Emerging Technologies.” They found in their study that uncivil comments can filter and distort reader understanding; but this was mostly true for those with strong prior beliefs or higher levels of religiosity. The news subject tested was nanotechnology, a kind of “blank slate” where the public is not particularly “primed” ideologically in one direction or another.

Popular Science online content director Suzanne LaBarre wrote: “If you carry out those results to their logical end — commenters shape public opinion; public opinion shapes public policy; public policy shapes how and whether and what research gets funded — you start to see why we feel compelled to hit the ‘off’ switch.” LaBarre then faced pushback from, among others, Columbia Journalism Review’s Alexis Sobel Fitts, who argued that the merely suggestive and circumscribed research findings don’t justify the Pop Sci editorial policy conclusion so tightly restricting comments.

Social Science Under-delivers for News Insight

So what do the authors of the study in question make of all this?

“I think Popular Science really should be commended for turning to research,” Scheufele said in an interview. “We are making so many decisions based on intuition, based on hunches — on what we think will work. We have increasingly slim margins in the news industry. The more we can rely on empirical research to make those decisions to guide us to a) commercially viable solutions and b) solutions that produce the best outcomes in terms of democratic citizenship, the better.”

Still, he maintains that “there’s not a one-size-fits-all solution based on our study.” And he also highlights a major research gap:

I would very much argue that social science has under-delivered on this topic. A lot of editors have asked us, “What is the other research that is out there? Is there other research?” And there really is not that much yet. I think this is social science not living up to its ability to do research on how we are going to navigate this new and rapidly changing information environment. The burden is really on us, and we really do need to respond to that challenge, and respond to that challenge in a timely fashion to be relevant to news organizations that are struggling.

Dietram Scheufele says social sciences research needed to meet pressing newsroom needs.

Scheufele — whose 2013 essay “Communicating Science in Social Settings” in Proceedings of the National Academy of Sciences (PNAS) highlights the more general problems for explaining science in any context — points to a rather striking fact about the communications research community. On a central issue of how the 21st century public square should be constructed and governed — how the Fourth Estate should moderate and encourage digital speech about issues before society — he says America’s sociologists, mass communications and journalism scholars, and political scientists have largely been absent.

News editors have simply eyeballed trends and guessed or randomly fiddled around with policy tweaks to improve comment threads. Sure, their Google Analytics can tell them something about unique page views and bounce rates; and these might be correlated with comment thread policies. But there remains much to learn about citizen knowledge and the wider civic good.

Brossard’s new literature review article in PNAS, “New Media Landscapes and the Science Information Consumer,” concludes: “In short, a well-written, balanced news story about a scientific topic may be interpreted differently because of the comments that follow it. Obviously, these results have important implications for science communication online and point to the importance of carefully evaluating how scientific information is presented, under which format, and with what type of contextualization.” She also notes that research in this area remains “scant.”

(As an aside, many news outlet audiences represent their own distinct, representative sample of the American public — skewing by income, race, geography, education, cultural values, etc. Although it makes general research findings all the more difficult, it may speak to the need for closer, embedded collaboration between social scientists and their regional news outlets.)

Research on Anonymous Comments

A few other studies also stand out.

In July 2013, Arthur Santana, a former newspaper reporter now on the communications faculty at the University of Houston, published a study in Journalism Practice that found requiring users to register under their real names reduces incivility in comment forums. He coded and analyzed content relating to immigration issues from a sample of news outlets that allowed anonymous postings in 2010 — Los Angeles Times, Arizona Republic and Houston Chronicle. Santana then compared the results with an analysis of a group of newspapers not allowing anonymity: Buffalo Daily News, Statesman Journal (Salem, OR), Burlington Free Press (Burlington, VT), USA Today, Detroit Free Press, Indianapolis Star, San Jose Mercury News, El Paso Times, Hartford Courant, Wall Street Journal, and Los Angeles Times blogs.

Santana concluded that anonymous comments are more likely to be uncivil: “Alarming is the finding that more than half of the comments in the anonymous forums following news on immigration were uncivil, meaning they contained some sentiment of hate, xenophobia or bigotry, thus representing a flagrant violation of the guidelines that online newspapers set for themselves.” By contrast, only about 29 percent of comments on sites with registered users were uncivil.

Other recent studies complement these findings: “Opinion Expression during Social Conflict: Comparing Online Reader Comments and Letters to the Editor,” published in Journalism by Jay Hmielowski and Michael McCluskey of Ohio State University; “Discussions in the Comments Section: Factors Influencing Participation and Interactivity in Online Newspapers’ Reader Comments,” published in New Media & Society by Patrick Weber of the University of Zurich; and “That’s Not the Way It Is: How User-Generated Comments on the News Affect Perceived Media Bias,” published in The Journal of Computer-Mediated Communication by Eun-Ju Lee of Seoul National University.

Still, there are those who argue anonymity has a strong value in the digital age — that enforcing “real names” policies is to be lamented, on any Web platform — particularly for those who may lack institutional support and may be vulnerable to online attacks for expressing their views.

Reporter/Editor Engagement — The ‘Gold Standard’?

The New York Times chooses 17 articles each day that will feature an open reader comment section. An editorial team is dedicated to moderating each comment, a luxury that few other outlets can afford. But many smaller outlets do have “community engagement” managers and editors who do some of this kind of work.

Natalie Jomini Stroud finds that reporter involvement in moderating content can lead to less incivility.

Research does suggest that a policy of editorial staff moderation may help achieve some balance between gatekeeper iron curtains and a verbal free-for-all.

A report from the Engaging News Project, from the Annette Strauss Institute for Civic Life at the University of Texas at Austin, finds preliminary evidence that comment moderation by reporters can pay off. That conclusion fits with the experiences of many newsroom reporters and editors. It is an approach practiced by many prominent bloggers in the climate change and science journalism communities.

“We’re only beginning to understand best practices, but our research with one local news station found that reporter involvement in the comment section resulted in lower levels of incivility,” Natalie Jomini Stroud, a scholar who oversees the Engaging News Project, said in an e-mail interview. “The reporter didn’t cure incivility, but the study does suggest that there are ways of reducing incivility. Perhaps other practices, such as badges, eliminating anonymity, or highlighting strong comments, also reduce uncivil commenting and can help to create a productive discussion space where citizens learn from each other.”

The Commenting Future: Rollback, Refinement — or Retrograde?

As news organizations have entered the digital age, two fundamental news audience-related decisions have colored much of media culture.

The first was the decision, originating in the late 1990s, to give away content for free, conditioning expectations among consumers. It’s an issue still being researched and debated — and some call it an “original sin.” As this decision continues to be revisited, more news paywalls have been erected as media outlets rethink digital age economics.

The other big decision, less obviously consequential from a business standpoint, was the opening up of news Web platforms to readers. Between 2007 and 2008, the percentage of the top 100 newspapers that allowed online comments went from 33 to 75 percent. It may be no coincidence that this transformation took place at precisely the time that Twitter first began to attract notice and Facebook was taking off. In other words, open comment forums were of a piece with the historical ethos of “Web 2.0” — a cultural shift toward an ethic of digital openness and “platform” thinking.

Further movement toward news outlet paywalls could reshape audience engagement, as many visitors would already be logged-in, registered, paying consumers.

Meanwhile, as research and media editorial policy struggle to catch up, the Web is changing toward an inchoate “3.0” stage, and social media engagement with news content has taken off. Many articles are boiled down and remixed into 140 character nuggets and “tweets,” with context, facts and quotes lifted out, and content is often being unbundled from the static page and the traditional news article format.

Smarter algorithms that can do more sophisticated “sentiment analysis” and other innovations around what’s being called the “semantic Web” might be enlisted to surface and feature certain kinds of audience comments. One can even imagine these algorithmically gathered comments as an active, endlessly flowing part of any news story, in the way reporters have traditionally gathered quotations from people through “shoe-leather” interviewing and newsgathering. Google has begun pioneering the use of machine learning and algorithmic curation of social media to help better order and surface relevant comments on YouTube posts. It suggests a future where all comments are “personalized” for individual visitors.

Even if social science catches up with the relatively “old” problem of comment thread trolls — and as journalists over time settle on best practices — the speed of the digital world may already be complicating things further for issues such as improved public understanding of climate science and science generally.