Without a significant improvement in the accuracy of seat polling, it would be foolish to predict any outcome

Why we need to take Longman and Braddon byelection polling with a grain of salt

There has been a rush of polling in some of the seats where byelections will be held this Saturday. We have seen multiple companies doing polling, particularly in the seats of Braddon and Longman, and a lot of conclusions have been drawn from this polling. But history suggests these polls should be taken with more than a few grains of salt.

Polls have generally been very close in the Queensland seat of Longman and the Tasmanian seat of Braddon, which are the two byelections where Labor and the Coalition are in contention.

We have seen four local polls in Braddon, ranging from 54% of the two-party preferred vote for the Liberal candidate, Brett Whiteley, to 52% for the Labor candidate, Justine Keay. The most recent poll, a YouGov/Galaxy poll commissioned by News Corp tabloids, predicted a tie with both candidates on 50% after preferences.

We have seen six polls in Longman. Five of these polls have had the LNP candidate, Trevor Ruthenberg, leading, with between 51% and 53% of the two-party preferred vote, with one other poll predicting a tie. The three most recent polls have all put Ruthenberg on 51%.

How much weight can we put in the polls? Unfortunately, not very much.

The race is much clearer in the South Australian seat of Mayo, where there have been four polls. All of them have had the Centre Alliance’s Rebekha Sharkie well out in front, polling 58%-62% of the two-candidate preferred vote over the Liberal candidate, Georgina Downer.

How much weight can we put in the polls? Unfortunately, not very much. Recent research by Simon Jackman and Luke Mansillo at the University of Sydney suggests that individual seat polling has a wide margin of error and cannot do a good job at predicting close races, based on the polling conducted before the 2016 federal election.

Australian polls do a good job of capturing the national mood but local electorate polls were often out by wide margins. Polls should ideally fall within the margin of error of the actual result in 95% of cases, yet pre-election polling in 2016 only managed to predict the primary vote for Labor and Coalition candidates 53% and 58% of the time respectively. This suggests that these local seat polls should have margins of error much larger than the published margins.

Facebook Twitter Pinterest Malcolm and Lucy Turnbull campaign with the LNP candidate for Longman, Trevor Ruthenberg. Five of six polls have had Ruthenberg leading in the seat. Photograph: Darren England/AAP

It is hard to know why these local polls have these problems but the pattern was similar for all polling companies.

This does not mean that seat polling is useless but they should be read in conjunction with other sources of information, and we should not expect them to be precise. A very clear result can be predictive, and they can tell us other things which don’t require a precise answer (for example, which issues voters care about), but we shouldn’t call a race based on a handful of polls giving one candidate 51% or 52% of the two-party preferred vote.

We already knew that Braddon and Longman would likely be close, based on the results in 2016, when Labor narrowly gained these seats off the Coalition. The recent polls confirm that these seats are likely to be close and suggest that there is a real possibility of Labor losing these seats. But it would be foolish to try and predict the winner based on a set of such close polls.

Longman and Braddon on a knife-edge ahead of super Saturday byelections Read more

The polling is much more decisive in Mayo, which suggests that Sharkie is likely to be re-elected, but the recent history of seat polling suggests these polls should still be treated cautiously.

These byelections are very unusual, and it is hard to find a precedent to help predict an outcome. It is very rare for the government to gain a seat at a byelection and MPs who lose their seats due to a violation of section 44 of the constitution usually win their seat back with ease. But it is also unusual to have byelections in very marginal seats, and quite strange to have sitting MPs resigning and re-contesting their own seats.

The polls confirm the sense that the races for Braddon and Longman are close, and that either major party could win either seat, but without a significant improvement in seat polling accuracy, it would be foolish to confidently predict any outcome before Saturday.