Sandvine: Netflix, Google ISP Rankings 'Conflicting,' Inaccurate

Network gear manufacturer Sandvine apparently isn't a big fan of both Netflix's and YouTube's new ISP streaming performance rankings, insisting that the data collected by both is unreliable and conflicting. In a blog post, Sandvine points out that ISPs deemed "HD Verified" by Google's new ISP ranking (discussed by us here) are sometimes categorized as under-performers in Netflix's rankings, and vice-versa:

I've already pointed out numerous times that Netflix's streaming rankings are made less valuable by the fact that Netflix CDN partners do better in the rankings than those who refuse to participate in the free service. Similarly I've noted Google and YouTube's new ISP rankings need some work as well, as navigation is clunky and some ISPs are listed as HD verified even in markets they don't even provide service.

With that said, it's worth noting that the lion's share of Sandvine's money comes courtesy of the nation's largest ISPs, and as such Sandvine has traditionally been reluctant to admit fault on the ISP end of the equation. The company recently proclaimed that YouTube streaming issues that have plagued most broadband users are entirely YouTube's fault, even if data tends to suggest there's plenty of blame to go around.

It also seems obvious that ranking YouTube and Netflix streaming performance would be "conflicting," given those rankings are testing entirely different content services, taking entirely different routes to the end user. Routes can be even more different now that companies like Netflix are (begrudgingly) striking direct interconnection deals with ISPs like Comcast and Verizon.

While both Google and Netflix efforts need work (and will hopefully improve), they're at least attempting to shed a little more light on the buffering problems that have plagued the industry the last few years. That's more than can be said for most ISPs, which have long hidden performance data (unless it's to cherry pick it to justify policies like usage caps, deregulation, etc.). If ISPs want to contest YouTube and Netflix's findings, they could simply release raw performance data of their own .

At a time when transit companies are claiming these slowdowns are intentional to make an extra buck , it seems like every shred of additional performance data is helpful if we're interested in the truth.