Although internet search engines like Google play an increasingly prominent role shaping voter opinions and perception of issues and candidates, their ranking algorithms aren’t designed to provide a fairly balanced or completely honest representation of controversial issues.

In order to assess how fairly search engine results portray political candidates and controversial issues, we collected over 1,200 URLs ranking highly in Google.com for politically-charged keywords such as “gun control”, “abortion”, “TPP”, and “Black Lives Matter”. Each URL was then assessed for political slant by politically active individuals from both the left and right. Finally, we used CanIRank’s SEO software to analyze how each URL compared in dozens of different ranking factors to determine whether Google’s algorithm treated websites similarly regardless of their political slant.

Among our key findings were that top search results were almost 40% more likely to contain pages with a “Left” or “Far Left” slant than they were pages from the right. Moreover, 16% of political keywords contained no right-leaning pages at all within the first page of results.

Our analysis of the algorithmic metrics underpinning those rankings suggests that factors within the Google algorithm itself may make it easier for sites with a left-leaning or centrist viewpoint to rank higher in Google search results compared to sites with a politically conservative viewpoint.

Why Web Search Matters to Democracy

This fall, Gallup reported that Americans’ trust in mass media had hit an all time low, with fewer than a third of the population saying they have a great deal or a fair amount of trust in mass media — such as newspapers, TV, and radio.

Perhaps a direct result of this distrust in traditional mass media, Americans are increasingly turning to online sources for their information.

According to recent Google findings, online search is the resource that 87% of the population turns to first when a question arises.

Online search plays a particularly prominent role in the democratic process during election season. During the 2012 election cycle, a survey of persuadable voters revealed that 49% get their news about campaigns and the election online, largely through search engines like Google, and that these voters generally trust the information they find online.

Top search results are broadly perceived as being the most accurate and authoritative by members of the public with the first five search results accounting for an estimated 67% of all clicks and the first three results alone accounting for over 55% of all clicks.

In their 2015 study, Robert Epstein and Ronald Robertson concluded that the order of search results can have a big impact on voter behavior — and in the event of a close election, this effect could even be profound enough to determine the outcome of the election.

The extent to which a single private company now controls the flow of information is unprecedented in a country historically characterized by pluralistic and ideologically diverse media. This trend is particularly troubling when one considers that the employees of this private company do not reflect the ideological diversity of the country at large, and have consistently been amongst the largest donors to Democratic party candidates.

Should Search Engines Be Required to Give Equal Time?

Fairly presenting both sides of an issue has been considered essential to a functioning democracy since the very early years, when America’s founding fathers used subsidies to support the development and survival of a diverse media corps. Since 1927, radio and later television broadcast stations have been required to give equal time to opposing political candidates. For many years, the Fairness Doctrine required media to present both sides of controversial issues in an honest and equitable manner, a standard that many journalists still aspire to.

Now that web search has largely superseded radio and television as the primary source of information for most Americans, should web search results also be expected to present both sides of an issue?

In our sample set of over 2,000 search results, we found that searchers are 39% more likely to be presented information with a left-leaning bias than they are information from the right.

But for some keywords, the search results are even more egregious. Does it make sense, for example, that someone researching “Republican platform” should be presented only the official text of the platform and seven left-leaning results highly critical of that platform, with zero results supporting it?

For other controversial keywords like “minimum wage”, “abortion,” “NAFTA”, “Iraq war”, “campaign finance reform”, “global warming”, “marijuana legalization”, and “tpp”, no right-leaning websites are to be found among the top results.

Search results for keywords like “Hillary Clinton seizures” and “Hillary Clinton sick”, on the other hand, were dominated by right-leaning websites and YouTube footage.

The proportion of results with a left-leaning bias increased for top ranking results, which typically receive the majority of clicks. For example, we found that search results denoted as demonstrating a left or far left slant received 40% more exposure in the top 3 ranking spots than search results considered to have a right or far right political slant.

Content with a left or far left viewpoint receives even more exposure in the top five search results. Searchers are 65% more likely to encounter liberal search results than they are conservative search results if they look at the first five returns to their query. Assuming typical click distributions, that means only 1 out of 10 searchers would end up clicking on a result with a conservative viewpoint.

Why do sites from the left feature more prominently in Google’s search results?

Google considers about 200 major ranking factors in their algorithm to determine which results to serve a user in response to a search query. In order to calculate accurate keyword difficulty scores, our software collects and analyzes many of those same factors, allowing us to gain some insight into why certain pages rank where they do. Of these many factors, Google has identified links (i.e., the number and quality of links pointing to a page) and content (i.e., relevancy and comprehensiveness) to be the two most important ranking factors.

Average number of external links

Based on previously documented correlations between specific ranking factor metrics analyzed by CanIRank and actual search rankings, we would expect top ranked search results to have more external links compared to lower ranked search results.

Instead, pages demonstrating a left or far left political slant made it into the top results with significantly fewer external links compared to pages rated balanced. Pages with a right-leaning slant needed significantly more links to make it into the top results.

Median content length

Content length is another metric that tends to correlate with higher rankings, as it indicates a greater depth of information on a subject. For example, small businesses and startups are often able to rank well despite having fewer links, if they provide more comprehensive content on a specific topic.

Among our sample list of search results, right-leaning pages had a higher median number of words across all 40 results compared to both neutral and left-leaning pages. Despite this, neutral and left-leaning pages represented a greater proportion of top search results.

Conclusion

In addition to external links and content length, we analyzed a host of other ranking factors including:

keyword usage in title, body, headings, img alt tags, and other HTML tags

related term usage in each of the above areas

number of indexed pages of the ranking website

relevancy of indexed pages to keyword topic

relevancy of home page

relevancy of the ranking website’s domain name

number of external and internal links to page and website

MozRank, MozTrust

Page Authority and Website Authority

Facebook shares/ likes/ comments, tweets, +1s

No discernible pattern emerged when looking at many of these factors. The evidence, we conclude, is not strong enough to prove an inherent bias within Google’s algorithm.

Though Google would like to portray itself as a fair and balanced arbiter of facts — a role it has recently tried to strengthen with the launch of a fact checking mechanism — searchers should be aware that ranking algorithms don’t currently incorporate an assessment of political bias or even factual accuracy. No attempt is made to present multiple viewpoints on controversial political issues, and the algorithm in its current form does not return results equally distributed across the entire political spectrum.

Methodology

To ascertain whether Google search results showed a political bias, we created a list of 50 politically-relevant keywords, which together receive over 20 million monthly searches.

After harvesting the URLs from the first 4 pages of Google.com’s search results for each keyword (with personalization features turned off), and then discarding any URLs where we were unable to collect data (mostly due to the limitations of Moz’s Linkscape index, our source for inbound link metrics), we were left with 1,207 URLs.

Next, we invited scorers from both sides of the political spectrum to assign a political slant score on a 1-5 scale to each individual URL, where:

1 represents content that demonstrates a heavy bias toward views aligned with the Democrat party or a progressive ideology

represents content that demonstrates a heavy bias toward views aligned with the Democrat party or a progressive ideology 2 represents content that demonstrates some bias toward views aligned with the Democrat party or a progressive ideology

represents content that demonstrates some bias toward views aligned with the Democrat party or a progressive ideology 3 represent content that is neutral or balanced

represent content that is neutral or balanced 4 represents content that demonstrates some bias toward views aligned with the Republican party or a conservative ideology

represents content that demonstrates some bias toward views aligned with the Republican party or a conservative ideology 5 represents content that demonstrates a heavy bias toward views aligned with the Republican party or a conservative ideology

Despite the large divide in our current political climate, our scorers were in agreement as to the degree and direction of bias for about half of the results, and were within 1 point on approximately 90%. In order to determine a consensus score for our analysis, we used an average of the political slant scores. The results were largely consistent with the political bias scores determined using a different methodology in a previous Facebook Research study.

In order to better understand why each URL ranks where it does, we used CanIRank’s SEO Competitive Analysis software to collect data on over 200 ranking factors for each URL ranking in the Google search results for these keywords.

The data collected by CanIRank’s software is intended to assess the relative strengths of a given URL for each of the major ranking factors that determine the ordering of results by internet search engines. Previous research has shown that CanIRank’s assessment of a URL’s competitiveness has an approximately 60% rank order correlation with the results ranking on Google.com. Most metrics analyzed by CanIRank fall into one of 7 major Ranking Factor areas:

Page Relevancy : relevancy of the specific page to the keyword

: relevancy of the specific page to the keyword Website Relevancy : relevancy of the entire website to the keyword

: relevancy of the entire website to the keyword Page External Relevancy : relevancy and strength of links to the specific URL

: relevancy and strength of links to the specific URL Website External Relevancy : relevancy and strength of links to the entire website

: relevancy and strength of links to the entire website Page Strength : trust and authority of the specific URL

: trust and authority of the specific URL Website Strength : trust and authority of the entire website

: trust and authority of the entire website Social: popularity of the page on social networks such as Facebook, Twitter, Google+, LinkedIn, etc.

Examples of some of the data points we collected and analyzed are:

keyword usage in title, body, headings, img alt tags, and other HTML tags

related term usage in each of the above areas

content length

# indexed pages from each domain

relevancy of indexed pages to keyword topic

relevancy of home page

relevancy of domain name

# external and internal links to the ranking page and website overall

Moz’s MozRank, Page Authority and Domain Authority

Facebook shares/ likes/ comments, tweets, Google +1s, LinkedIn shares, Pinterest Pins

Finally, we assessed the strength of individual and aggregated ranking metrics relative to the URL’s actual ranking for each of our 5 political slant scores (Far Left, Left, Center, Right, Far Right). Although Google’s search ranking algorithm is non-linear and third-party metrics are at best a rough approximation of the data they use internally, in general and in aggregate, higher ranking results score higher in most ranking factor metrics. In an algorithm with no political bias, we would expect this pattern to hold up regardless of the political slant of the specific result.