Last October, Google promised that it was stepping up efforts to fight online piracy of TV and movie content. A new ranking algorithm would downrank illegitimate sources. New ad and “Knowledge Graph” units would direct people to legitimate sources. Six months later, the launch of the latest season of HBO’s Game Of Thrones TV series shows that Google is doing well in some respects but still has more work to do in others. The same is true for Microsoft’s Bing search engine.

Game Of Thrones is a useful case study because of its past history of being one of the most pirated shows online. A key factor has been that many people have had no legitimate way to purchase the most current season online. HBO Go provides this but only to those with HBO already through cable subscriptions. As a result, those without cable might dig deeper into the search results, seeking episodes from the current season, legitimate or not.

This season is different. HBO’s new HBO Now service provides a legitimate way for anyone to watch Game Of Thrones, if they’re willing to pay for it. I was curious about how the new service and Google’s promised changes might impact Google’s results for Game Of Thrones. After all, plenty always turn to Google to find GoT content.

This season is no different, with Game Of Thrones being the top trending topic on Google after its premiere last Sunday:

Among the search topics, “game of thrones” was obviously popular. So was “game of thrones seasons 5” and “game of thrones season 5 episode 1.” Almost certainly, topics like “watch game of thrones” and “game of thrones download” were being used, as Google is targeting terms like “watch” and “download” along with TV show names as part of Google’s anti-piracy efforts.

So let’s run some of those searches and see what we get.

“Game Of Thrones” Results Are Piracy Free

Here are the results for a search on “game of thrones” as I saw them this week, logged out from Google and using Incognito mode on Chrome, so as to avoid as much personalization as possible:

Google does well here in returning HBO itself as the top web listing after all the news headlines. There are no red flags in terms of piracy for the other links. Wikipedia, HBO’s own Facebook page for Game Of Thrones and news articles that make up most of the rest of the listings all seem fine. You can even see that at the bottom of the page, Google discloses that it has removed a page due to a piracy complaint, which looks like it was related to someone copying IMDB content.

By the way, I’d say HBO needs to do a better job trying to convert people who follow its main link in Google’s results to purchase an HBO subscription. The link leads to the aforementioned HBO Go service, designed for those who already have HBO. The landing page for HBO Go doesn’t make the new HBO Now option for newcomers very prominent, in my opinion:

Unhelpful Knowledge Graph Box

Where Google’s search results arguably go wrong is in terms of new units designed to also stem piracy. On the right side of the page is what’s called a Knowledge Graph box. Within it are three ads, new units that Google introduced to help content owners buy prominent placement.

HBO didn’t buy an ad here. Of those who did, Vudu, Google Play and Amazon, none of them have the current episodes, which are reasonably what many people searching for “game of thrones” might be looking for.

It’s fair to say that if HBO is concerned with piracy, it failed to exercise enough effort to place ads for HBO Now in this area. But it’s also equally fair to say that as a search engine, Google’s job is to direct people to the best information regardless of having been paid for it. Google doesn’t do that here. In contrast, it did make a special effort of this nature when the Apple Watch was announced.

There are more ads at the bottom of the page, part of Google’s regular AdWords system:

These seem OK. Working from the bottom, the Amazon ad is pretty clear that it only offers seasons 1-4. There’s no fault with this. The ShareTV ad is generally OK. It doesn’t promise that you can watch Game Of Thrones. Clicking and going to the ShareTV site lists places where you can watch Game Of Thrones, along with other information. The Yidio site, when you arrive at it, is a little more aggressive in pushing full episode availability. But like ShareTV, it points at legitimate sources and YouTube clips from HBO.

Overall, I’d give Google’s search results in this case an A- grade. Piracy is kept out, as Google has promised. The results could be better with some effort by HBO. However, Google itself could have made them better with more work on its end to promote HBO Now.

Bing has made no explicit promises in terms of fighting piracy, in large part because due to its much smaller share of search activity, it doesn’t get targeted much by Hollywood. But I thought it would be useful to look at Bing as well as Google, for a better industry picture. At Bing, the results are about the same as Google in terms of cleanness and usefulness. HBO Now doesn’t have any visibility, but HBO Go does.

Getting Specific & A Bing Failure

I also ran searches for “game of thrones seasons 5” and “game of thrones seasons 5 episode 1,” as both were popular, according to Google Trends. In both cases, I found no piracy issues.

For the first, there was no Knowledge Graph Box that appeared. HBO Go was the second web listing on the page. Apple had the top ad running, pitching HBO Now. For the second, there was again no piracy issues. However, HBO wasn’t listed at all. Apple was still there with the second ad on the page, pitching HBO Now. There was a small show-specific Knowledge Graph box that wasn’t that helpful.

I did the same at Bing. For “game of thrones seasons 5,” I spotted no piracy issues, but HBO Go didn’t get listed until midway down the page. Bing also provided a huge Knowledge Graph-like box for this search. For “game of thrones seasons 5 episode 1,” the top listing lead directly to a pirated copy apparently fed out of Box and for season 1, episode 5:

Horrible Sites In The Piracy-Prone Searches

Google’s promised changes were especially meant to help with specific searches involving terms often used by those seeking or unknowingly ending-up at pirated content. A phrase like “watch game of thrones” would be one of these, using an example from Google’s own post about the changes. How do the results for that fare?

The results begin with the special ad unit. Once again, HBO itself apparently has chosen not to participate in that:

After the ads and a news box come the main web page listings:

Most of these have no piracy issues. But the first two — which come directly above HBO itself — are terrible. Both purport to have copies of episodes that haven’t even aired yet, which might be due to how coming episodes have leaked online. I can’t tell if the sites actually have them or not. That’s because they freak me out with all the windows and pop-ups and things they try to make you install.

Both sites are a horrible experience for any searcher to arrive at, whether they actually have pirated copies or not. Neither should be getting listed, much less listed above HBO itself. I’d give Google a D grade for these results.

That first horrible site above also is the first site listed for “watch game of thrones free,” another query Google’s supposed to be targeting. The rest of the results are clean, but HBO itself doesn’t show. I give these results a D- grade, since the lack of HBO is an issue.

For “game of thrones download,” another query with a term often associated with piracy, the results are better. However, there was still a link at the bottom of the results to one of those terrible sites like the ones mentioned above that purport to offer free copies but which trigger a bunch of horrible pop-ups, if you try any of the viewing links.

Why should legitimate sources show up simply because they paid, when the user experience should be that such sites get listed regardless?

I’d give these results a C. At least the horrible link is fairly buried. But HBO isn’t listed first. And why should legitimate sources show up simply because they paid, when the user experience should be that such sites get listed regardless?

As for Bing, “watch game of thrones” brings up one of those terrible sites first, above HBO. I give it the same D grade Google got, for this. For “watch game of thrones free,” there’s a horrible site at the top, another midway down and no HBO. A D- grade for this. For “game of thrones download,” like Google, a horrible site listed first, then HBO, then the rest of the result are piracy free. A C grade for this.

Overall: Not As Bad As Expected But Could Be Better

I actually started looking into how Google was handling these types of searches back in January, just before The Walking Dead started its mid-season run. At the time, the results for something like “watch walking dead online” were horrible.

Those results did have an ad box with trustworthy sources at the top; AMC’s official channel was listed first in the free web results, but there was no Knowledge Graph box Google like had promised. Worse, the web results were loaded with those aforementioned horrible sites that try to get you download stuff and seem generally unsafe.

Looking again this week, the results are the same mess:

Of the ten sites listed, the four I’ve pointed out feel simply unsafe for people to visit. Potentially, they also have pirated content. As for the others, there’s not a lot of value-add to listing them. Since AMC actually offers the last half of the current season for free, I kind of felt like Google would be doing a big benefit just listing AMC along with ads and nothing else. Or better, do some work to list the legitimate providers of episodes, no ads required.

Google’s response when I asked about this in January was that the ads are official sources, that Google works to protect users from malicious sites and to point me to a paper on how it fights piracy. None of that felt satisfactory to me. Piracy issues aside, Google’s search results felt like crap. As a search engine that’s supposed to value relevancy, I expect it to do better than this. The same is true for Bing.

I do appreciate the wackamole nature of dealing with these types of sites. I also appreciate that for the far more popular searches of TV shows just by name, rather than with qualifiers like “watch” and “download,” the results seem much cleaner. I also appreciate that for my latest look above with Game Of Thrones, I didn’t find as much of a mess as with The Walking Dead.

I also appreciate that it’s hard to point people to legitimate places when sometimes no legitimate places that exist. This is true for any Hollywood movie that comes out and isn’t released on digital until the theatrical “window” has closed, typically 90 days. Now that HBO Now exists, HBO itself certainly has a much better official place to point people to — and it clearly needs to ramp up some of its own SEO and SEM efforts to drive them there. It shouldn’t be hard.

Running Ads Isn’t The Complete Solution

But ultimately, what Google promised last year could be better. It feels like Google relies too much on its ads to direct people to legitimate sources, which was never what Google was supposed to be about. To quote from Google’s own origin story, the academic paper that cofounders Larry Page and Sergey Brin wrote in 1998:

Advertising income often provides an incentive to provide poor quality search results. For example, we noticed a major search engine would not return a large airline’s homepage when the airline’s name was given as a query. It so happened that the airline had placed an expensive ad, linked to the query that was its name. A better search engine would not have required this ad, and possibly resulted in the loss of the revenue from the airline to the search engine.

Google was supposed to be that better search engine. Running ads in addition to listing free legitimate sources would be meeting that goal. Running ads without listing the legitimate sources falls short of that.

As for Bing, it has promised nothing at all. That doesn’t mean it shouldn’t aspire to the same standards that Google has promised. It has plenty of work to do, too.