Opinion

No one seems to be a fan of Tumblr’s adult content ban. It’s hurting users who turned to Tumblr for nonmainstream porn. It’s harming the sex workers and adult artists that fostered its popularity. It’s sending the message to marginalized communities that their work and desires don’t matter. But this is only the beginning. The way Tumblr plans to moderate NSFW content spells trouble for the internet’s future.

For starters, Tumblr’s current algorithm is abysmal. Tumblr users have been astonished with its sheer number of false positives over the past week, with SFW content regularly labeled as explicit. Twitter has since created the hashtag #TooSexyForTumblr and posted the site’s false flags. These include dragons, pets, Minecraft characters, boot scrubbing designs, puking unicorns, dinosaurs wearing flower crowns, holiday sweaters, and Vice President Joe Biden.

Tumblr thinks my Totoro design is just TOO HOT TO HANDLE!!!!



Get your own xxx TOTORO xxx ugly holiday sweater and steam up yoour holiday party in style 🤣✨🌱🍃https://t.co/ZPlCmRN7j5



🌲❄️🌳🌱❄️🌿❄️✨❄️🌙❄️⭐️#UglyChristmasSweater #myneighbortotoro #TOOSEXYFORTUMBLR pic.twitter.com/IyxCujZEzP — Annie Stoll ⁷ 🗡✨🌹🌈💜🦋🌸 (@aniistoll) December 4, 2018

hell yes! my puking unicorn got flagged #tooSexyForTumblr

nice work @tumblr , you blew it pic.twitter.com/s9CYuqTR5H — a.j. (@stewped) December 4, 2018

Joe Biden is #TooSexyForTumblr I guess. pic.twitter.com/iMuVokSVsp — I will not go quietly into the night (@ofborrowedlight) December 4, 2018

But this kind of disastrous NSFW algorithm—that threatens sex workers, adult content creators, and queer users who struggle to find an affirming 18+ community to call home—is likely the new standard for machine-based content moderation for porn.

During 2018’s third quarter, Facebook took down 30.8 million pieces of adult content, with 95.9 percent caught “before users reported it” thanks to the site’s detection technology. In an apparently recent move, Facebook’s Community Guidelines now ban “vague suggestive statements,” “sexual hints,” and “explicit sexual solicitation,” as well as drawings that depict “explicit sexual activity” or “suggestively posed persons.”

First Tumblr bans NSFW imagery, now Facebook bans NSFW *words*!! pic.twitter.com/ozb0nwgEUr — Vivian (@suchnerve) December 5, 2018

Puritanical attitudes are strengthening once again in the United States. We’re sliding backward into the days where just discussing sex resulted in formal censure. — Vivian (@suchnerve) December 5, 2018

The USA may not have formally banned sexual content from the Internet, but the recent passage of SESTA/FOSTA and the rhetoric around them have told people all they need to know about what their rulers think of sex outside married, white, heterosexual reproduction. — Vivian (@suchnerve) December 5, 2018

Facebook isn’t the only social media platform that relies on machine learning. In 2015, Twitter acquired the AI company Madbits, which developed a visual intelligence system that automatically identifies pornographic content. Twitter now relies on machine learning for everything from organizing users’ timelines to finding relevant search results. While Twitter allows NSFW content, it’s easy to imagine how quickly Twitter could remove adult work if it decided to censor its platform.

For adult artists, sex workers, and queer users leaving Tumblr amid its NSFW crackdown, the site’s drastic policy change is a reminder that sexual content is only safe to the extent that it is profitable. Discord pulled the plug on NSFW content in Partnered Servers just one year after receiving a $50 million investment round. And in Tumblr’s case, one anonymous former staff engineer suggested the NSFW ban’s roots come from Verizon, Tumblr’s parent company, after the phone company struggled to sell ads next to pornographic content on the site, according to Vox.

Not just that, but future NSFW moderation may be open-sourced, letting communities work together to implement adult content filters that censor sex workers. In 2016, Tumblr’s owner Yahoo published its algorithm “open_nsfw” on GitHub, letting any internet user study a deep neural network that “takes an image as input and outputs a probability which can be used to detect and filter NSFW images.”

Gabriel Goh, a University of California, Davis graduate student, was able to use open_nsfw to generate random images with a visualization technique that activates classifier neurons to study how open_nsfw identifies NSFW content. It’s a bit like reverse engineering an algorithm to see how it thinks—and it’s creepily good at its job. As filmmaker Tim Nicholas accurately describes it, the generated photos are “totally abstract yet distinctly pornographic, like HR Geiger or something.”

Fascinating look at how Tumblr uses neural networks to detect porn – when asked to generate “NSFW” images, the pictures it generates are totally abstract yet distinctly pornographic, like HR Geiger or something: https://t.co/BAeaGL4Bwj — Joachim of Biore (@tim_nich) December 5, 2018

According to Yahoo open nsfw, this is the most not safe for work picture that the mars curiosity rover has taken so far, with a score of 96%. pic.twitter.com/IEsimSEAlE — Pim de Groot (@mifune) December 3, 2018

Tumblr most likely uses open nsfw, thinks it is porn and will delete this mars curiosity picture the 17th. pic.twitter.com/yLXPEZMb6Z — Pim de Groot (@mifune) December 4, 2018

READ MORE

While it’s unclear if open_nsfw is the same algorithm behind Tumblr’s NSFW bans, it isn’t hard to imagine a dedicated development team building off the open-source algorithm. In the near future, NSFW content algorithms could be as readily available as SEO plugins, letting social media platforms exile sex workers from day one with advanced image recognition technology.

In a world increasingly infatuated with controlling, identifying, and removing adult content, it’s hard to know which start-up social media platforms will actively care about protecting sex workers. As Tumblr’s Dec. 17 deadline to remove NSFW content looms closer, a replacement run by and for sex workers is needed. Only then will social media have adult creators’ true interests at heart.