Since Donald Trump announced his presidential run in 2016, his list of adversaries seems to grow endlessly, from career politicians to celebrities to media outlets. But this past Tuesday, that list saw an unusual addition: Google. Trump angrily touted on Twitter that the dominant search engine had “RIGGED” its search results by shutting out rightwing media and directing users to content from “Fake News” leftwing media. But behind the bombast, does he have a point? Are search engine results biased? And if they were, how would we know?

The role of social media like Facebook and Twitter in politics has been intensely debated since the 2016 elections, and those conversations have continued as the 2018 midterm elections approach. But while Facebook suffered intense scrutiny culminating in CEO Mark Zuckerberg’s testifying before Congress, Google and other search engines have remained relatively unscathed. This is perhaps unsurprising; at first glance, it might appear that search engines like Google are fundamentally different from news or social media – simply a way for web users to access existing information, rather than a media source in their own right.

This way of thinking about search results is wrong. Recent studies suggest that search engines, rather than providing a neutral way to find information, may actually play a major role in shaping public opinion on political issues and candidates. Some research has even argued that search results can affect the outcomes of close elections. In a study aptly titled In Google We Trust participants heavily prioritized the first page of search results, and the order of the results on that page, and continued to do so even when researchers reversed the order of the actual results.

Sign up to receive the latest US opinion pieces every weekday

Search results, in our view, can have the same kind of impact on people as stories reported by a news outlet or the opinions shared on a social media site. We argue that search results should be considered a form of media. We call it search media: the set of search results and their ordering shown to users.

If we think of search as media, then it’s not surprising that Google has been caught in the fray. If search results are consistently biased against a candidate or party, the political implications could be very real. The question, then, is what it would mean for search media to be “rigged” or biased. Trump, in his tweets, argues without any evidence that Google’s search results are deliberately biased against him by the company itself, something that, if true, would obviously constitute rigging.

When Google’s search engine first launched in 1997, it started as a well-documented academic project. Its now-famous algorithm, PageRank, was totally transparent for anyone to know. This had some unintended side-effects, since spammers and other self-serving actors took advantage of their knowledge to try to trick Google into prioritizing their content. After years of the ensuing arms race, with Google making secret changes to the algorithm and spammers trying to game it, the result is that Google’s search algorithm is a behemoth whose inner workings are a near-complete mystery outside the company. Such a complex and proprietary system makes Trump’s claim of anti-conservative suppression difficult to prove or disprove. The secrecy with which these algorithms are shrouded makes them a prime target for fear-mongering claims like Trump’s.

Given the unlikelihood of Google publicly revealing the details of its secret algorithm, then, what options do we have for studying search media and their effects? Rather than investigating its interior processes, we can instead study the output of these algorithms. One way we can do this is by categorizing each result in the search list according to a project from Harvard that identifies media sources by their partisanship based on share patterns on Twitter. If we search for “Trump news” on Google (using a computer address in Washington DC), we can see why Trump may have been upset. Of the nine results returned, eight of the results are center-left (NYT, CNN, BBC, Time, USA Today) while one is on the right (Fox News).

There are two problems, however, with this kind of analysis. The first is that Google’s search engine is returning results from popular news organizations with established reputations and journalists trained in the practices of objective journalism. The Google search engine was designed from the outset to surface information from reliable sources, regardless of political orientation, so these results shouldn’t surprise us. But the other problem is that it isn’t sufficient to look for bias at a single point in time for a single keyword. Instead, we must systematically monitor search results for a broad range of searches over time. Our team at Stanford is doing exactly that, focusing on Google search results about all candidates running in the 2018 midterm elections.

Search media are more than ever a significant source of information for citizens, with studies showing that people are more likely to find and trust news through search than social media sites. As with other forms of media, they are likely to affect our democratic process in significant and often unpredictable ways. Trump is right about one thing: search engines, like other media sources, have some power over the information that citizens seek. Analyzing search media systematically and over time, as we do with other forms of media, is critical to understanding whether search media are biased and what impact they might have on our democracy.