In our last post, we saw that there are some pretty strong relationships between the Special Elections Index and the House popular vote, just as good as, if not better than, using generic congressional ballot polling averages. A trendline can, of course, be easily turned into a prediction (as seen above, where we use the Special Election Index values for the year prior to election year)—but there’s some uncertainty to pay attention to (shown as the green range).

To summarize, you want to have at least 50 election pairs (which we have already exceeded for the 2017-2018 biennium) and preferably more like 100; the index value should settle down after the November prior to the election year but may still drift; and watch out for a cluster of extreme elections from one state. Finally, the Index can predict good years and bad years, but the uncertainty is great enough that the prediction should be thought of more as a range than a specific number.

Practical considerations

The Special Elections Index was constructed using all available data. That means special elections were compared to other elections that happened both before and after the specials. But when making a prediction, elections from the future obviously aren’t available. What about predictions made with data available at the time?

Here’s a graph that shows the biennial Special Elections Index, combining data for both years of each election cycle, recalculated using only data from past elections for comparison.

It still looks pretty good, actually, when compared to the same plot using both prior and later elections:

There’s increased scatter in the first graph, but that’s probably just because we now have fewer election pairs included in most years. In fact, a plot of absolute error versus number of election pairs shows the same general pattern for both types of calculations, those including all elections and those only using past elections: more error with fewer data. That error plot actually gives us a good empirical basis for estimating the range of errors to expect when trying to make a prediction. Practically speaking, this also means the Special Elections Index will be much less useful in the biennium after the first elections with new boundaries from redistricting (2014, for example), when far fewer prior elections are available for comparison.

Watch for stability

The trend lines were calculated based on data for at least a full year. So can we use them for a partial year?

Sure! Just like we watch polls, even though it’s only the polls conducted close to election day that we want to look at when we evaluate accuracy. As long as there’s enough data, we can follow the Special Elections Index (and its predictions) to look for trends. Just so long as we understand the final numbers aren’t really in yet.

Here’s an example of 2017 and three recent cycles with enough data to look at (only past elections are used for this graph, so it’s what we would have seen at the time):

In the beginning, there’s a lot of movement, as there are very few special elections early in each year, so each additional election represents a substantial share of the total. But by the fall, when we typically have more than 50 election pairs, things have settled more. Then there’s a big batch in November (because many special elections are consolidated with regularly scheduled November elections), which caused a shift in 2009, 2011, and 2017. At this point in each cycle, it takes a lot of change to move the Special Elections Index, but we still see both 2008 and 2012 drifting upwards until the final special election.

2017 stabilized pretty early into a range of 12 to 14; the November special elections brought that number down to about nine.

Outliers

We finally turn our attention to 1998. Over and over, poor 1998 has been excluded and ignored. Why is that?

1998 is based on only 28 election pairs. That’s a low number to start with, so we already know we have a lot of error. But more than half of the election pairs are from just three special elections, all in districts on Long Island in New York, and all with absolutely ghastly numbers for Democrats (Long Island was once one of the most rock-ribbed GOP strongholds in the country). It’s clear they have undue influence over the 1998 data; just as clear, without them there are too few election pairs left to mean anything. It’s a lose-lose situation for data analysts.

This points to something we should be wary of more generally: one state with unusual circumstances and a ton of special elections. In 2017, for instance, Oklahoma had an exceptionally unpopular Republican governor on top of the generally poor national environment for Republicans, and five of the special elections there have had simply gaudy numbers for Democrats. Is that warping the Special Elections Index for 2017? Fortunately, the answer is probably not, since we have a good mix of states to work with; in fact, the Oklahoma numbers almost (but not quite) balance out the horrid numbers coming out of Connecticut (where Democrats are weighed down by their own deeply unpopular governor).

Speaking of which: There is the strange case of Connecticut. Special elections in Connecticut tend to see huge changes from regular elections. Typically, they’re terrible for Democrats—although occasionally, sometimes when Democrats are having a really bad year nationwide, the Democratic numbers soar in a Connecticut special election. See that huge dip early in 2009 in the graph above? That’s due to six special elections in Connecticut slamming into the index. Eliminating all Connecticut data actually improves all the correlations. I just don’t know what it is about Connecticut.

What it all means for 2018

We already have enough data to confidently say things are looking really good for Democrats. If the Special Elections Index functions this cycle as it has in past cycles, there’s a real chance we will see the best Democratic year in decades, with the popular House vote likely to be within 5 points of 2008 levels. But will it be enough to win back the House? At current values of the Index, at least, there would be a reasonable chance, even though the consensus is that Democrats will need a substantial advantage in the popular vote in order to take back the House, with a typical estimate being around seven points.

Data

For a detailed description of the Special Elections Index and its relationship to election outcomes, see prior posts (here and here). All elections data available here, including Special Elections Index values.