As mentioned above, we have a huge number of races with third party candidates this year, and they don't seem to be fading away as election day approaches. In the past, this has led to large errors in the polling averages—two-thirds of the time, more than four points in the margin. Here is what that looks like for the closest races:



The green circles show races where third party candidates ended up with more than 5 percent of the vote, combined. (This is roughly equivalent to 6 percent in polls.) It's clear that most of them are quite far from the diagonal line where an ideal polling average should lie; indeed, among the farthest. The gray points show all the rest of the data.

Another source of error in polling averages is too few polls. This isn't much of an issue in close races, which are usually polled frequently, unless the race is changing faster than the pollsters are polling. Below, the yellow circles have 10 or more polls in the average, while the green circles have fewer than five.



There's not many green circles; however, half of them are a fair distance away from the perfection of the diagonal line. When you look at all races, not just close ones, it's quite clear that errors are related to the number of polls in the polling average.

Once we narrow down the data set to just those polling averages with five or more polls, and no races with third party candidates getting more than 5 percent, we can find a familiar pattern based on the partisan lean of the state showing up.

Below, the blue circles show only the elections in states where President Obama won more than 55 percent of the vote in 2008:



In every case but two, polls underestimated the Democratic candidate. The dashed line shows a regression through the blue data points, including points beyond the range of the graph. The polling average underestimated the Democratic margin by about three points on average for these data points.

Below, the mirror image: elections in states where Obama won less than 45 percent of the vote in 2008:

The most striking difference is how few data points there are. We've essentially been playing the political game on our home turf for the past ten years.with close Senate races in Alaska, Arkansas, Louisiana, Kentucky, Kansas and, ever so briefly, South Dakota. We also have had close races for governor in Alaska, Arizona, Arkansas and Kansas.

As you can see from the graph, these races are more likely than not to have polls that are too friendly toward the Democratic candidates. On average, however, it is only a one-point error. Ancestrally Democratic states such as Arkansas, Kentucky, and Louisiana might not consistently show this effect, however.

A practical guide:

Here's a summary of what the last ten years of polling errors can tell us.

1. Look at the number of polls you have. If you have only a handful of polls, your polling average will likely have much greater error.

2. Look for third party candidates. If you have a third party candidate polling around 6 percent or so, the polling average is likely to be off by five or more points on the margin. This is true for many races this year.

3. If you have plenty of polls and no or low-polling third party candidates, look at the state. Deep blue states will, on average, have polls that are too Republican by a few points, while deep red states, on average, will have polls that are too Democratic by a point.

4. Finally, remember that past performance is no guarantee of future results.

Note: The polling averages used in this post are the averages of the margins of all polls from October 1 to Election Day, unless a trend was observed, in which case only polls from the final 10 days of the election were used.

Update: The title was changed to make it clear that the polls can be wrong either way.