Facebook’s challenge is getting its technology to work well enough that its roughly 15,000 human reviewers can reliably pick up the slack, in each of the more than 100 countries and languages the service is used. Getting its hate speech and bullying detectors close to the effectiveness and autonomy of its porn filters will be particularly difficult.

Deep learning algorithms are pretty good at sorting images into categories—cat or car, porn or not porn. They’ve also made computers better with language, enabling virtual assistants like Alexa and significant jumps in the accuracy of automatic translations. But they’re still a long way from understanding even relatively simple text in the way humans do.

Decoding Language

To understand whether a post reading “I’m going to beat you” is a threat or a friendly joke, a human reviewer might effortlessly take into account whether it was paired with an image of a neighborhood basketball court, or the phrasing and tone of earlier messages. “How a model could use context in that way is not understood,” says Ruihong Huang, a professor at Texas A&M University. She helped organize an academic workshop on using algorithms to fight online abuse this fall, at one of the world’s top conferences for language processing research. Attendance and the number of papers submitted roughly doubled compared with the event’s debut in 2017—and not because researchers smelled victory. “Many companies and people in academia are realizing this is an important task and problem, but the progress is not that satisfying so far,” says Huang. “The current models are not that intelligent in short, that’s the problem.”

Srinivas Narayanan, who leads engineering in Facebook’s Applied Machine Learning group, agrees. He’s proud of the work his team has done on systems that can scan for porn and hate speech at huge scale, but human-level accuracy and nuance remains a distant hope. “I think we’re still far away from being able to understand that deeply,” he says. “I think machines can eventually, but we just don’t know how.”

Facebook has a large, multinational AI lab working on long term, fundamental research that may one day help solve that mystery. It also has journalists, lawmakers, civil society groups, and even the UN expecting improvements right now. Facebook’s AI team needs to develop tricks that can deliver meaningful progress before the next scandal hits.

The products of that push for practical new AI tools include a system called Rosetta announced this year that reads out text that is embedded in images and video, allowing it to be fed into hate speech detectors. (There’s evidence some online trolls are already testing ways to trick it.) Another project used billions of hashtags from Instagram users to improve Facebook’s image recognition systems. The company has even used examples of bullying posts on Facebook to train a kind of AI-powered cyberbully, which generates text generator to push its moderation algorithms to get better. The company declined to provide WIRED a sample of its output.

One big challenge for these projects is that today’s machine learning algorithms must be trained with narrow, specific data. This summer, Facebook changed how some of its human moderators work, in part to generate more useful training data on hate speech. Instead of using their knowledge of Facebook’s rules to decide whether to delete a post flagged for hate speech, workers answered a series of narrower questions. Did the post use a slur? Does it make reference to a protected category? Was that category attacked in this post? A reviewer could then scan through all the answers to make the final call. The responses are also useful feedstock for training algorithms to spot slurs or other things for themselves. “That granular labeling gets us really exciting raw training data to build out classifiers,” says Aashin Gautam, who leads a team that develops content moderation processes. Facebook is exploring making this new model permanent, initially for hate speech, and then perhaps for other categories of prohibited content.

LEARN MORE The WIRED Guide to Artificial Intelligence

Elsewhere, Facebook is trying to sidestep the training data problem. One lesson from the tragic events in Myanmar is that the company needs to get better at putting humans and software in place to understand the language and culture of different markets, says Justin Osofsky, a vice president who runs global operations.