Freedom of speech has traditionally been an issue of government and human rights. But more and more companies are providing platforms where anyone can potentially contribute some sort of speech, typically text. And those companies are finding that they face many of the same issues governments have: how to balance giving users the ability to express themselves freely against the possibility that they'll post problematic content.

"Problematic" has various definitions. In some cases, it's truly dangerous, like incitements to violence or false medical advice. And companies may find that they don't want to be associated with expressions of racism, sexism, or other forms of prejudice. But can companies do anything if people use their service for broadcasting content that the companies don't approve of?

A new study answers that question with a clear "yes." Researchers looked at Reddit's fight against hate speech, which saw it ban a variety of subreddits in 2015. The analysis suggests that the regular users of these subreddits toned down their language as they moved to other areas on the site. And a number of users who wanted to continue to share offensive opinions simply went to other services, making them someone else's problem.

Further Reading reddit says it will actively ban harassing subreddits, axes its first five

Back in 2015, Reddit announced that it would begin banning subreddits that "allow their communities to use the subreddit as a platform to harass individuals." Two of the obvious targets for this, r/fatpeoplehate and r/CoonTown, which targeted the obese and black people, were banned shortly afterward. The new study, published in the Proceedings of the ACM on Human-Computer Interaction, looks at what happened after the ban, using public information from Reddit for some detailed data mining.

Big data

First, the researchers built a database of over 100 million posts and comments from 2015, as the bans occurred around mid-year. They then used two methods to identify potential hate speech. One mined the contents of fatpeoplehate and CoonTown to look for terms that were uniquely used in those subreddits. In addition, they manually curated this list, selecting terms from it that were clearly offensive. Although neither of these lists would match everyone's definition of hate speech, both of them do provide at least a functional indication of speech associated with hatred, or at least trolling.

"These lexicons [of hate speech] are publicly available to the community as a resource," the authors note. Presumably, you could also consult them if you wanted to hate like a pro or describe other people's hate ("shitlording" appears in the list).

To understand the behavior of the users of the two subreddits in question, the authors needed a control population. They constructed one by identifying the other subreddits that the users posted in and finding a collection of users who had a similar posting pattern but didn't frequent the sections of the site that were banned.

With that, they tracked how user behavior changed after the ban. For many users, the end of these two subreddits meant the end of their time at Reddit. Once fatpeoplehate was shut down, 21 percent of the accounts went inactive, and another 12 were deleted entirely. That compares to the normal churn in the control group, where a bit over 10 percent went inactive, and another 11 percent deleted their accounts. For CoonTown, the change was even more dramatic: 19 percent went inactive, and 21 percent deleted their account. The control population here was a bit more likely to leave Reddit, though, with 16 percent going inactive and 12 percent deleting their accounts.

That doesn't mean the users gave up on hate, however. A look at a site that allowed its users to continue engaging in racism and fat-shaming called Voat showed that over 1,500 Reddit usernames showed up on that site.

Behavioral change you can believe in

The majority of users of the two banned subreddits, however, continued to use Reddit and generally continued to post with similar frequencies before and after the ban. Were those users simply engaging in the same behavior elsewhere?

There's some indication they tried. For the first few weeks after the ban, the use of hate speech terms dropped, but it experienced sudden spikes. These likely reflect attempts by the individuals to re-form new communities, which Reddit generally identified and banned as well, the authors suggest.

By about 50 days after the original ban, however, things settled down considerably. In the case of the manually curated list of terms, the former subscribers of fatpeoplehate essentially stopped using them, becoming indistinguishable from the control users. Ex-CoonTowners continued to use some racist terms, but at a fraction of the frequency they had while the subreddit was open. In both cases, the fully automated list of hate terms saw that former users of banned subreddits continued to use hate terms at a higher frequency than controls, but far less often than they used to.

The former users of fatpeoplehate ended up moving on to a variety of subreddits. The only one that was clearly focused on their former habit of insulting strangers is called RoastMe, in which people post pictures of themselves to be mocked. Some of CoonTown's former inhabitants also migrated there, but a fair number moved on to The_Donald, homeland, and BlackCrimeMatters, subreddits where "racist behavior has either been noted or is prevalent," as the authors put it. Still, their racist speech dropped considerably, even in these environments where it might be tolerated by other users.

FTW

From Reddit's perspective, the aggressive moderation worked, as hate speech on the site dropped. While some evidence suggests that people who are truly dedicated to denigrating their fellow humans migrated to other sites, these sites don't have the prominence of Reddit, so the presence of hate speech there has less effect on the targets of these users' disdain.

Some of these individuals are undoubtedly present among the people who abandoned or deleted their accounts following the crackdown. But it's also likely that additional people quit as a form of protest against Reddit getting aggressive with its moderation for the first time. It's also possible that the people who quit Reddit included individuals who reveled in being offensive but weren't committed to the specific hatred that these subreddits pursued. The extent to which people abandoned Reddit because they couldn't pursue their racism or other forms of hatred can't be determined from this data.

But the key finding is that the people who stuck around changed their behavior, conforming their language to the norms of the new subreddits they became active in. Part of that may have been fear of seeing another hangout end up banned; that same fear may have motivated moderators of remaining subreddits to be more aggressive about policing language. But it's possible that a few of them figured out that attacking someone for their appearance wasn't socially acceptable in general.

As the authors noted, this wasn't a guaranteed result; there are examples in the literature of aggressive moderation being ineffective or leading some communities to spiral out of control. Plenty of other companies are also facing challenges with figuring out how to provide an open platform while preventing threats of violence or overt discrimination. Having an example of successful moderation from a community as large and factious as Reddit may make the case that other sites can intervene successfully.

Full disclosure: the author regularly intervenes to moderate discussions at this website.

Proceedings of the ACM on Human-Computer Interaction, 2017. DOI: 10.1145/3134666 (About DOIs).