Second Thoughts: Polls Are Bad, But How Bad Are They?

by Ryan Lambert/Columnist (@twolinepass)

Yes, polls are bad. But how bad are they?

One of the most common refrains college hockey fans hear throughout the season — and should take to heart — is that polls are bad. Or stupid. Or some other negative descriptor that is an accurate reflection of how seriously you should take them, which is to say, “not at all.”

No one puts enough thought into them to really earn the credence people give them; it’s the kind of thing you fill out on a Sunday evening between the 4:15 NFL games and Sunday Night Football by seeing who won over the past few days and, maybe, who they beat.

We can assure you here and now that the opinions of pundits, coaches, and others who vote in these kinds of things should be taken with a grain of salt at the very best of times. Frankly, most polls — especially as the season progresses — are just a list of teams with high winning percentages, arranged roughly by how frequently they win with some adjustments for strength of conference mixed in.

The only poll that really ought to matter to anyone for any reason, it seems to me, is the preseason poll. Instead of being a kind of casual rearranging of the Pairwise coming out of any given weekend, it should be people who have a real and deep knowledge of the NCAA hockey landscape taking time to really think about where things stand. In theory, this should at least somewhat accurately reflect how things will shake out at the end of the year.

So with the acknowledgement that even preseason polls are bad and worrying about where your favorite team happens to fall in them is a total waste of time, you do have to wonder just how pointless they are. Or at least, you do if you’re me. And being me, I ran the numbers.

The question is: “How predictive are the annual preseason polls?”

The answer is: “Barely.”

Dating back to 2006-07 (that’s the farthest back full polls, including “Others receiving votes” go, as far as I could find quickly and easily), the USCHO poll saw between 33 and 42 teams receive at least one vote to finish in the top 20. If that sounds like throwing darts blindfolded, you’re not that far off. Only about two-thirds of the teams to receive a vote in any preseason poll over the last 13 years — not counting this season’s, for obvious reasons — ended the year in the top 20 in the Pairwise.

The accuracy was even lower for teams ranked in the top 16 to start the year actually finishing there: 63.5 percent. Teams the voters thought would finish in the top 10? Less than 55 percent. Top five? Just 38.5 percent.

And obviously, all this comes with the word of warning that hockey is the most random sport to predict, especially in short seasons like the NCAA’s. Playing something like 34-38 games before the Pairwise is solidified doesn’t really give you absolute best outcomes possible. And often, teams that were predicted to finish in the top 10 will still find themselves in the top 20 or so.

However, the average top-10 team in the preseason over a period of 13 years was overrated by about six spots. What’s really amazing is that for every team that received even a single vote to finish in the top 20 — again, between 33 and 42 teams, but an average of close to 37, which is close to two-thirds of the entire NCAA — almost 1 in 4 didn’t even end the season falling into a range that broad.

Altogether, there was only about a 31 percent correlation between where teams were ranked to start the year and where they ended up. That number dropped precipitously (to as little as 3.3 percent) when you narrowed the field to top-20, top-16, and top-10.

Let’s put it this way: No team picked to finish first since 2006-07 has actually ended the season occupying that spot in the Pairwise. And just as many preseason No. 1s finished 18th as finished second. Only two won national titles, and last year’s Minnesota Duluth team was a bit of a gimme if we’re being honest.

None of this, by the way, is to say that the people putting these polls together are doing a bad job or anything. It’s a random sport, and voters are going to be overly deferential to teams that had success the previous season, regardless of how realistic building on that success actually is. Even if you’re making your absolute best guesses instead of half-assing it, it would be difficult to project all this to a level any random fan would find acceptable.

So yes, polls are stupid. Could they be less stupid? Sure. Could they be significantly less stupid? Probably not.

You’re always going to be better off ignoring it than giving it any credence. Especially if your team started the year in the top 5.

Ryan Lambert is a Yahoo! Sports columnist. His email is here and his Twitter is here.