Recommendation Vulnerability: Alternative Influence

As I said before, YouTube is not just a search engine, it’s also a recommendation engine. When you watch a video on YouTube, you are recommended a slew of “related” videos to watch. More importantly, once you finish a video, a new one auto-plays out of those recommendations. This is fantastic if you’re listening to music. You end up getting exposed to all sorts of new and interesting songs. But what if you’re listening to political commentary? What if you’re listening to news-y content?

Recommendation engines and auto-play features are designed to encourage you to continue to engage. The algorithms behind these features are constantly “improved” to maximize that outcome. The designers of these systems don’t pay attention to the content itself, but instead pay attention to a set of signals, trying to determine what makes something relevant to one person or another. For example, if someone watches Video A and then watches Video B (or, better yet, actively likes or comments on both videos), the next person who watches Video A will likely be recommended Video B. Pretty simple, right? But of course, that relationship is something that can be exploited. After all, making those types of connections isn’t something that the news community thinks about. They put the videos up, they leave the videos there.

The communities that are trying to shape these connections understand how to produce connections.

Consider what happens if you’re searching for vaccination information on YouTube. The first content you will almost always get is factual, high-quality video content. Many health organizations, including the Center for Disease Control, have responsibly produced videos that are available on YouTube to detail how vaccination is safe and important and to explain the conspiracy around anti-vaccination culture. Their videos are, let’s be honest, a little dry, but they are still well-produced and informative. However, the conspiratorial anti-vaxx community is hell-bent on getting their message of doubt through to parents who might be wavering, who might be beginning to search for information on vaccines. And many search engines have struggled with the sophisticated SEO practices that the anti-vaxx conspiracy groups use.

YouTube has a different problem. If you watch a health organization’s video and then you follow the recommendations you’re given or allow auto-play to continue, within two videos you will almost always be watching a conspiracy video. Why? Because the communities that are trying to shape these connections understand how to produce connections. They know that comments matter, so they comment on both videos. They know that links matter. So, they help shape links. They know views matter, so they get their community to watch both videos. They strategically and intentionally train the algorithms to build a link between CDC content and anti-vaxxing messages. This means that when people search YouTube for videos about vaccination, they are highly likely to be exposed to anti-vaxxing messages.

My colleague Becca Lewis described another method to influence news content in her report, Alternative Influence. While the CDC will never host an anti-vaxxer for a “debate” about vaccination, many people in the political context use this format. They think it’s appropriate to create a false equivalency, to have two people debating. This occurs in mainstream news, but it also occurs on YouTube. How it plays out on YouTube, though, really matters. Mainstream commentators host people who take extreme views on their channels to debate. In doing so, they send a signal to YouTube that these channels should be linked in the recommendation system. And so, the next people who are watching the mainstream channel will likely be recommended the one hosted by someone with fringe perspectives. This is a way of creating a pathway, a connection. It’s a way of manipulating the actual network graph of these systems. You go to YouTube for reasonably informed information and within a few recommendations, you are exposed to fringe, extremist, or conspiratorial content.

When people first hear about this dynamic, their initial response is — well, remove extreme content. And I understand that sentiment. They want YouTube to not allow anti-vaxx material. Or hateful or conspiratorial content. I have a lot of sympathy for that response, but there’s a problem if you think about the long run. YouTube has already gotten rid of a LOT of content, it has “down weighted” a lot of content, and there’s still so much there that’s utterly terrible. Partially this is because creators who have an agenda, like those in the anti-vaxx communities, have learned to skirt the lines. After all, most anti-vaxx content doesn’t tell people to not vaccinate; it asks people to question whether or not vaccinations are safe. That’s the process: seeding doubt. And it’s much harder to talk about removal with content focused on doubt. YouTube is especially sensitive to this because they don’t want to be seen as politically biased or removing content that is trying to promote dialogue.

They don’t send hateful messages. They just get their audience to doubt common ideas.

Of course, some media manipulators know how to exploit companies’ anxiety around political censorship to push the edge and promote anti-scientific frames. How many of you watch PragerU videos? How many of you are familiar with PragerU?

PragerU is produced by Dennis Prager, a conservative talk show host. PragerU produces a video a week. Their goal is to undo the “leftist” messages produced by universities. Their videos are popular on YouTube, but they are especially popular on Facebook. Their CEO claims that one third of all U.S. Facebook users has watched a PragerU video on Facebook. That’s significant. If you talk to people who are teaching in universities right now, they are constantly getting questions that come from students watching PragerU.

PragerU exploits data voids on YouTube to invite people to doubt widespread values. They don’t send hateful messages. They just get their audience to doubt common ideas. For example, if you’re a teenager who just encountered the term “social justice,” you might throw it into YouTube. If you do, you won’t get a conversation about the history of the term or the different movements involved in it or why a commitment to addressing historical inequities is important. Instead, you’ll get a PragerU video telling you that “social justice” is a propagandist term, that the term is not meant to help you, but is actually meant to harm you. (Of course, who is included and excluded in this “you” is significant given that their target audience is often conservative and religious. Whether they mean to or not, they help encourage young white men to see themselves as the “real” oppressed people.) PragerU’s strategy works because “social justice” is another data void: racial justice movements have left the term behind and are no longer producing new content related to “social justice.”

Once you watch one PragerU video, you’ll be given a non-stop stream of them. There are hundreds of them. Maybe you’ll get a video titled “Why No One Trusts the Mainstream Media.” Or the one on “What They Haven’t Told You About Climate Change.” Let’s check out the latter.

A screen capture showing how YouTube includes a link to Wikipedia on PragerU’s video about global warming.

YouTube recognizes it’s in contested territory so it provides a link to a Wikipedia entry. But why the entry on “global warming” instead of “climate change?” I don’t know. “Global warming” is another left-behind term. So, if you search for “global warming” (which is a reasonable thing to do on YouTube), you’re going to get hoax videos. Climate change denial videos. YouTube helped climate deniers build that pathway.

The problem with these paths is that most of them are not total disinformation. They are arguments for doubting a particular line of thought. They are inviting you to question, to see doubt. To look for more information — to do your research. And if YouTube removes such content, no matter how conspiratorial, they are met with charges of censorship. It’s a very familiar strategy. The same thing occurred with Russia Today (RT), and their “Question More” campaign in the UK.

Russia Today poster questioning climate change.

RT put posters around the UK: “Is climate change more science fiction than science fact?” They seem to be inviting you to question more, to consume all sides of the news. When the UK responded by removing them, RT put up their next round of ads: “This is What Happens. Redacted! Censorship!” In this way, they staged a challenge to speech. They seeded doubt and when that was called out as propaganda, they were required to remove the posters, at which point they decried that their rights were being taken away.

I’m not convinced that removing conspiratorial or doubtful content actually gets us anywhere in the long run; it simply creates different types of polarization. Do I believe some types of content need to go? Absolutely. But we need to think about where the power is in this dynamic. How do we understand the link between curiosity and extremism?

How do we understand the link between curiosity and extremism?

There are other ways to approach this. Personally, I’m fond of a technique that Spotify has implemented. I don’t know how many of you use Spotify, but you know how when you listen to something and you’re in a groove, and then suddenly you’re interrupted and you’re like not that. There are certain things that are known to be massive disruptions; for example, take the Christmas music problem. No matter how much you love Mariah Carey, if you’re grooving out to “We Belong Together,” you don’t want to follow it with “All I Want for Christmas is You.” If you’ve chosen to listen to top 40 hits, you don’t want to be slammed with a Christmas album. If you’re listening to Christmas music, it’s because you’re already in the Christmas music thing. And once you’re there, all you want is Christmas music. It’s a separate universe. Same thing with kids’ music. You may like They Might Be Giants, but you don’t want their kids album. So, Spotify has had to actively break structural patterns in their data that are sensible according to many criteria. And that requires understanding content, and really understanding context. On YouTube, it’s more complicated, but I’d argue that the company needs to actively examine and break certain recommendations by recognizing the strategic deployment of doubt and conspiratorial thinking on their platform. To realize that recommendations are fundamentally about amplification and to think responsibly about their role as amplifiers.

Epistemological Vulnerability: Scriptural Inference

Of course, any choice to design algorithms to amplify content or shape what people might see raises a different problem. Who decides what should be amplified and what shouldn’t be? What’s conspiratorial and what’s legitimate difference of opinion? When should disagreements be bridged and when should people not be exposed to different perspectives? My commitments as a social scientist mean that I believe it’s unethical to show people climate denial content when they’re looking to learn about climate science. The same is true for anti-vaxxer and Holocaust denial content. Suggesting that these are “sides” to a debate is a form of false equivalency that I believe is dangerous and irresponsible.

Yet, the lines aren’t always clear. As a researcher, I also recognize that people hold different and often contradictory truths. I respect that one person’s religion is another’s myth. I recognize that political commitments are exceptionally nuanced. And when it comes to knowledge, I accept that people hold different epistemologies. In other words, how people know what they know varies. I may be wedded to rationality, evidence, and reason, but I respect that some people start with experience or faith.

Since Eli Pariser first coined the term, many people have lamented the presence of “filter bubbles” on social media. In the political context, they’re seen as dangerous, especially on sites like Facebook. What does it mean to not be exposed to “the other side”? Researchers struggle with this, because often people choose to self-segregate regardless of what algorithms recommend. They double down on a world that’s just like them. When is it appropriate for recommendation engines to expose people to new information? Is that helping inform people? Is that a form of proselytizing? Is that an act of power that needs to be contested? And what happens when the starting points are in two radically different places?

When communities focus on getting to the “right” search query, manipulators can help stage content that people who search for different terms never see.

In Searching for Alternative Facts, my colleague Francesca Tripodi describes sitting in a Bible study in Virginia. After spending an hour analyzing a Biblical passage, the pastor turned to talk about the then-new Tax Reform bill. Diving into a particular passage, the pastor encouraged his congregants to apply the same tools of scriptural sense-making that they applied to the Bible to this political text. Using this method of analysis, their interpretation diverged wildly from how political wonks read the same information.

Francesca goes on to describe how the practice of scriptural inference is also applied to Google searches. Instead of using the search engine to research a topic, many of the conservative Evangelicals she observed approached Google for clarity and affirmation of something that they had been told. Within this context, leaders in the community — from pastors to talk radio personalities — asked listeners to search for specific terms to confirm the truth. They’re not encouraged to read the articles or try different paths, but to construct the “right” search query so that Google can provide the “right” information. And if you use the “wrong” search query, you get the “wrong” information. When communities focus on getting to the “right” search query, manipulators can help stage content that people who search for different terms never see.

This is important because where people start from sends people down different information paths both because of the architecture of search and because of how people approach search differently. Consider the difference between searching on YouTube for “Vatican pedophiles” versus “Vatican homosexuality.”

Screen capture of YouTube results for search terms “vatican pedophiles,” “vatican sexual abuse,” and “vatican homosexuality.”

On one hand, it’s very responsible of YouTube that it keeps the search results for these two queries separate and distinct. After all, it’s a very dangerous thing to collapse “homosexuality” and “pedophiles” into one category. But on the other hand, this means that whichever of those search terms you use to investigate for information about recent scandals will send you into a completely different world of content. These terms send you down entirely different paths. Your information landscape, your recommendation engines, everything is shaped based on how you begin this process.

You’re not just interacting with misinformation through content. You’re dealing with it through all the surrounding information.

Talking to Republican voters during a primary, Francesca was curious to know how they determined which candidates to vote for. Everyone around her told her that they didn’t trust “fake news,” by which they meant mainstream news sources — usually symbolized by CNN. So, she knew they were going to find a different path that was not about news, per se. She was expecting she would hear about community and different ways of trying to make sense of things. But they were like, “No. We go to Google.” And she thought — “Oh, I’m an academic. I know how to Google. Why is this so problematic?” And then she realized something important. These voters didn’t search to go look at the content. They didn’t click on the links. They put side by side, all of the candidates in a primary race to see what Google would offer up, because Google provided “all sides.” And they felt this would provide perspective. In other words, those headlines, those little clips, served up as search results, they mattered more than any actual content. The same ends up being true on YouTube. So much of the information ecosystem there is not about watching videos, but about seeing the clips and comments that are surfaced during searches. The things that surround the videos, the text that is meant to get you to click on it — this text becomes the end of the story, not the beginning of it.

This was not how Google (or YouTube) was designed. When we talk to Google engineers about the fact that people are doing this, they flip out. Google isn’t designed to be liberal or conservative, but it is designed to be data-driven. It has an epistemology that assumes that the knowledge is within the data. And it is designed with the assumption that people will click through to that data, that content, not simply read the headlines. This is a fallacy that we struggle with in general. Those in the field of journalism know there’s a constant struggle over what headlines do. Are headlines trying to get people in to read, or do they do the work on their own? How is your worldview shaped through a collection of headlines written by editors who each stretch the story in order to drive clicks if people don’t actually click? And it’s not just that people only read the headlines, it’s also that people follow the world through the notifications they receive on their phone, pushed from Twitter or Facebook. You’re not just interacting with misinformation through content. You’re dealing with it through all the surrounding information. And that’s what manipulators know how to exploit.