Photo: Getty

Who is going to win the 2015 general election, now only five days away? The polls seem to be inconclusive. In the past 48 hours one put the Tories ahead 5, others put Labour ahead and another showed a tie.

But there is more consistency than meets the eye. There now appears to be a stark and critical difference between phone and online polls – as others have suggested before us. If we take an average of phone polls over the past fortnight, the Tories are more than three points ahead. If we take an average of online polls, the race is still tied.

This divide didn’t exist earlier this year. The data conflicted. This across-the-board methodological difference is recent and significant.

31 polls have been conducted in the past fortnight. 24 have been online, 7 have been by phone. As a group, the phone polls have put the Tories on 35.0 per cent and Labour on 31.6 per cent. The online ones have put the former on 33.4 per cent and latter on 33.7 per cent. In other words, online polls have put the Tories a point and a half lower, and Labour two points higher.

There are more online pollsters and they publish more polls. [1] They have published 77 per cent of the polls in our two-week polling average, which is why our election prediction shows a race closer to 34/33 than 35/32 or 35/31.

But that may be wrong. If the phone pollsters are right, Cameron has a good chance of retaining power this month. He could still face an anti-Tory majority even if he wins by three points – this was the key point of our election overview 13 days ago – but a 35/32 win would give Cameron a clear victory and hope.

Phone versus online

Is the divide as binary as phone versus online? It seems to be; it holds whether or not we exclude YouGov, who have conducted 10 of the 24 online polls over the past fortnight. [2]

Recall that, as a group, 24 online polls have given Labour a 33.7 to 33.4 per cent advantage over the past fortnight. Well, YouGov’s ten polls have given them a 34.6 to 33.8 advantage, and online pollsters excluding YouGov have given the Tories a 33.1 to 33.0 lead.

So you might suggest YouGov are in even greater opposition to phone pollsters than online polls as a group. But this difference in lead is slight and fades if we just considers the past five days. YouGov have given both parties 34.5 per cent across their four polls since Monday, with other online pollsters giving the Tories a 33.6 to 33.4 advantage. [3]

These online numbers all reinforce the idea that the two parties on are on 33-34 per cent, or that if the Tories are nearer to 35 per cent, so are Labour.

Only one in seven phone polls has showed a 35/35 race in the past fortnight.

In contrast, only one in seven phone polls has showed a 35/35 race in the past fortnight: the latest ComRes/Mail poll two night ago. The previous six – by Ashcroft, ICM, ComRes and Ipsos MORI – all put the Tories on 34-36 per cent and Labour on 30-32 per cent.

Has this divide existed surreptitiously for a while? We don’t think so. In the two weeks before the past fortnight, four phone polls were published. Labour averaged 33.5 per cent in them, and polled above 33 per cent in every one.

Similarly, the Tories polled between 33-34 per cent in three. They managed 39 per cent in a fourth but the Guardian cautioned against over-interpreting it at the time. Without that 39 per cent the figures were in line with the idea of a 33-34/33-34 race.

Ukip

So why is there now this difference? It seems to come down to Ukip.

In the past fortnight, Ukip have polled less than 11.5 per cent in phone polls.

Over both the past fortnight and the past five days, Ukip have polled less than 11.5 per cent (11.3) in phone polls. That’s three points lower than the 14.3 per cent online pollsters have given them in the past two weeks; and two and a half points lower than the 13.7 per cent online polls have given them this week.

YouGov do stand apart more here. They’ve given Ukip 12.8 per cent over the past fortnight and 12.0 per cent over the past five days – not much more than phone polls. But other online pollsters differ unmistakably; they show 15.4 per cent for Ukip over the past fortnight and 15.0 per cent this week.

This suggests that as Ukip fade, so the Tories’ rise. We can test this. Does that hold across all 31 polls in the past fortnight?

Yes – there is a fairly strong negative correlation between Ukip’s vote share and the Tories’. In other words, as Ukip’s vote share falls, the Tories’ share rises. (The closer an r value – highlighted on the right in the graphic – is to 1, the stronger the relationship; the p value shows whether something is statistically significant – it is if p is <0.05.)

This relationship holds if we consider far more polls. We only have 31 here.

But 341 have been published since this site launched on September 9, 2014 (an arbitrary but useful date; the election campaign broadly began then, as the Scottish referendum campaign concluded and party conferences began).

The link is consistent and statistically significant across all 341 polls. Ukip’s vote share explains around 40 per cent of the Tories’ poll rating, as shown by something called the r-squared value, highlighted in the graphic below.

That’s a noteworthy link, and it’s far stronger than the relationship between any two other parties (for instance the Greens’ vote share only explains around an eighth of Labour’s vote).

If we model the link between the Tory and Ukip shares, an 11 per cent Ukip poll hands the Tories nearly 35 per cent. But the link can easily be overstated: the Tory share can range from 32-37 per cent if Ukip poll 11 and still ‘fit’ the data.

And Ukip’s poll share does not correlate with the Tory lead over Labour, because Labour’s share is not linked to Ukip’s. Take YouGov: they show a tie but have Ukip as low as 12 per cent this week. Ukip’s share matters, but is still only part of the picture. (All regression analysis via the magical Statwing.)

Which polls are right?

If the polls continue to show this methodological divide, one form of polling is going to win on Thursday. Until then we can only theorise about who is right.

Many pundits seem to trust phone pollsters more. Two of the phone pollsters are the longest established firms – ICM, who poll for the Guardian, and Ipsos MORI, who poll for the Standard; a third is Lord Ashcroft, whose constituency polls and focus groups mean his weekly national poll has often had great impact; and the fourth is ComRes, who poll for the Mail.

Many pundits trust phone pollsters more, but are they missing ‘shy Ukippers’?

If we skip over our qualifiers and focus on Ukip, could phone polls be understating their support? You might suggest that no vote is more socially stigmatising than one for Ukip, and so perhaps phone polls are suffering from a ‘shy Ukipper’ effect.

Some poll-watchers are convinced that there are ‘shy Tories’ in this election, as in 1992, but David Cameron is widely considered to be a better possible PM than Ed Miliband, and the Tories have a double-digit lead on the economy.

This is all mere conjecture, but it’s not clear that a Tory would be embarrassed in the way they might have been in 1992; the coalition’s cuts are disliked by some, but they haven’t been as polarising as 13 years of Thatcherism (11 years, 6 months of Thatcher).

What about the idea that online polls are picking up too many enthusiastic Ukippers? Some think this happened during the Scottish referendum, when pollsters slightly overstated the likely Yes vote, and during ‘Cleggmania’ in 2010, when the pollsters all overstated the Lib Dem vote share after the TV debates. In other words, are polls consistently surveying too many Ukippers? This can’t be completely corrected by weighting after a certain point.

But the enthusiasm bias towards the SNP and Lib Dems wasn’t specific to online polls; enthusiastic supporters are as likely to pick up a phone quickly as they are to fill out a survey online. And Ukippers are one of the least likely groups to be online. They and the Tories are less likely to have shared something online during this campaign than other partisans, and yet they are crowding out online polls?

Maybe. We’ll see soon.

**

[1] There are six main online pollsters: YouGov (the Sun and Sunday Times’ pollster); Populus; Opinium (the Observer’s); Survation (the Mirror’s and Mail on Sunday’s); Panelbase; and TNS. Our data includes a seventh, BMG Research, who are a new entrant and whose debut poll we published this week.

[1] YouGov have recently started to poll every day, having polled five times a week since we launched in September. We do not track their two extra polls (conducted on Fridays and Saturdays); unfortunately, our system wasn’t built for one pollster to poll this much.

We keep an eye on these polls, but we think any shifts in YouGov’s numbers will be picked up in their other five polls a week.

[3] This data was completed before Survation’s poll last night, but that poll and YouGov’s Friday night results fit this analysis.