“We just haven’t seen comparable pressure for platforms to go after white violence,” and if they do, companies face “political blowback from the right,” says Bloch-Wehba. “It feeds into a narrative about who terrorists are, who is seen as a threat, and what kinds of violent content is presumed to be risky.”

Bloch-Wehba says tech companies' definitions of terrorism tend to be vague, but ISIS and al Qaeda are typically the only groups named in their transparency reports, which reveals their priorities.

The cycle is self-reinforcing: The companies collect more data on what ISIS content looks like based on law enforcement’s myopic and under-inclusive views, and then this skewed data is fed to surveillance systems, she says. Meanwhile, consumers don’t have enough visibility in the process to know whether these tools are proportionate to the threat, whether they filter too much content, or whether they discriminate against certain groups, she says.

If platforms are now having a harder time automating the process of identifying content from white nationalists or white supremacists, “it’s going to be hard for them to play catch-up,” Bloch-Wehba says.

Madihha Ahussain, special counsel for anti-Muslim bigotry for Muslim Advocates, says it’s not just a matter of expanding guidelines around terrorist content. Tech companies fail to enforce established community standards. “We believe there’s a lot of content generated from white nationalist groups generally that would violate” tech platform guidelines, but “it takes a lot on the part of advocacy groups to see some action.”

For years, Muslim Advocates took it as a good sign that tech executives would meet with the group and appeared responsive. “But then we realized that nothing was actually changing,” Ahussain says.

In a statement to WIRED, a YouTube spokesperson said, “Over the last few years we have heavily invested in human review teams and smart technology that helps us quickly detect, review, and remove this type of content. We have thousands of people around the world who review and counter abuse of our platforms and we encourage users to flag any videos that they believe violate our guidelines.”

YouTube says its guidelines prohibiting violent or graphic content that incites violence are not limited to foreign terrorist organizations and go beyond just ISIS and al Qaeda. The company estimates that the Global Internet Forum contained 100,000 hashes of known terrorist content at the end of 2018.

YouTube also says it’s taking a stricter approach to videos flagged by users that contain controversial religious or supremacist content, even if they don't violate the company’s guidelines. In such cases, YouTube will not allow the videos to contain ads, and it says it will remove the videos from its recommendations algorithms and remove features like comments, suggested videos, and likes.

In a statement, a spokesperson for Twitter said, “As per our Hateful Conduct Policy, we prohibit behavior that targets individuals based on protected categories including race, ethnicity, national origin or religious affiliation. This includes references to violent events where protected groups have been the primary targets or victims.”

Facebook pointed to a company blog post on Monday about its response to the New Zealand tragedy. The company said the original Facebook Live video was removed and hashed “so that other shares that are visually similar to that video are then detected and automatically removed from Facebook and Instagram.” Since variants of screen recordings of the stream were difficult to detect, Facebook used audio technology to detect additional copies.

Tech platforms have a financial interest in promoting their own version of “free expression,” Bloch-Wehba says. “Any attempt to move away comes laden with this set of assumptions about consumer rights, but those aren’t really legal rights—or, at least, they’re very unsettled legal rights,” she says. Nonetheless, “it plays into the same conversation, mostly coming to the right wing, that we should all be able to say whatever we want.”

Ahussain says meaningful change will only come if tech platforms want to address the issue, but the lack of diversity within tech companies has led to a lack of understanding about the complexities and nuances of threats faced by Muslims. To address that, Muslim Advocates and other groups want tech companies to hear directly from the communities that have been impacted. “We’ve recognized the need to have conversations in a neutral space,” and with a chance to set the tone, agenda, and guest list, she says.

More Great WIRED Stories