Show caption Andrew Gillum’s victory in the Democratic primary for Florida governor last month was enough to make pollsters look like liars. Photograph: Christopher Aluka Berry/Reuters US midterms 2018 Are polls broken? Democrats’ unforeseen wins pose urgent question Polling underestimated the strengths of certain Democratic candidates in a number of primaries this year – inaccuracies that could affect the midterm elections in November Tom McCarthy @TeeMcSee Mon 10 Sep 2018 10.51 EDT Share on Facebook

Share on Twitter

Share via Email

In the Gravis poll, he was fourth. In the Florida Atlantic University poll, he was tied for third. In the SurveyUSA poll, he was fourth again. And on election night, he was first, winning by tens of thousands of votes.

Andrew Gillum’s victory in the Democratic primary for Florida governor last month was enough to make pollsters look like liars. Some members of the public are already inclined to think that. Didn’t the pollsters also say Hillary Clinton was going to win in 2016 and that Britain would “remain” in Europe?

Among opponents of Donald Trump, as hopes swell that the midterm elections will send a blue wave crashing through the US Capitol, the question of whether polling is broken is an urgent one. The promise of victory can generate energy among voters and increase turnout, but in a number of primaries this year polling has underestimated the strengths of certain Democratic candidates – especially candidates of color.

The “surprise” victory of Gillum, who is African American, was preceded by a “surprise” win in New York by Alexandria Ocasio-Cortez, who is Hispanic, and followed by a “surprise” win in Massachusetts by Ayanna Pressley, who is also African American. Pressley was 13 points behind until she won by 17 – a 30-point miss for the pollsters for local WBUR. Ben Jealous and Stacey Abrams, non-white gubernatorial nominees in Maryland and Georgia, also beat the numbers.

If comparable blind spots persist through the general elections this November, the ability of Democrats to win the 13 or so toss-up races in the Senate, the 72 or so toss-up House races and the 18 toss-up gubernatorial races could be affected. The Democrats need to pick up 23 seats to gain control of the House and two to take the Senate.

Celinda Lake, president of Lake Research Partners, said any pollster whose 2018 election model was based on voter turnout in 2014 – before Trump, before “build the wall”, before “fake news”, before “both sides” – would miss the mark.

What the turnout factor is for millennials, what the turnout factor is for people of color, really has a massive impact

“The single hardest thing in the polling – and it’s turning out to be very complicated so far this year – is to figure out who’s actually going to turn out to vote,” she said. “And that’s what determined a lot of these surprises and these margins, and most recently particularly for candidates of color.

“What the turnout factor is for millennials, what the turnout factor is for people of color, really has a massive impact and it’s very often predicted incorrectly.”

Polling remains vital to interpreting elections. Certain supposed misses by pollsters in the past were not in fact misses. For example, national polls were on the money in the 2016 presidential election: Clinton was meant to win the popular vote by about three points. In fact she won it by two.

“It’s important to say that there’s really no question about how the [2016] polls did relative to historical standards,” said David Dutwin, president of the American Association for Public Opinion Research.

“It’s an empirical fact. The national polls actually were more accurate in the last election cycle than the average of the last 20, 30, 40 years of polling. At the state level there certainly were challenges, although a lot of those challenges had nothing to do with quality.”

The challenges for state pollsters included insufficient frequency, smaller sample sizes and less sophisticated likely voter models, Dutwin said.

“It would be nice for these things to be fixed. The problem I think is that just basically requires money. More extensive likely voter models cost money. Larger sample sizes cost money.”

G Terry Madonna is director of the Franklin & Marshall College Poll in Pennsylvania, which before the 2016 election measured Clinton as running ahead of Trump by double digits in the state. She ended up narrowly losing.

“In our case, we were out of the field 10 days before the election and we didn’t miss a significant change within the electorate that took place because we weren’t polling what was happening,” Madonna said, referring to factors including a decision by the then FBI director, James Comey, to publicize the reactivation of an investigation of Clinton’s emails.

“We also had a very unusual election. We had the two most unpopular candidates ever to be nominated by their respective political parties. Which is not to say that we don’t have to constantly work on improving our methodologies.”

If some media consumers are tempering their intake of polling information, some media producers are changing how they cover polls. In April the Associated Press added a new guideline to its stylebook, ruling that “poll results that seek to preview the outcome of an election must never be the lead, headline or single subject of any story”.

Public opinion experts are working, meanwhile, to educate people about how to judge the quality of polling information they see.

Barack Obama greets the crowd at a Democratic rally in Anaheim on Saturday. Photograph: Barbara Davidson/Getty Images

“All polling requires you to make some assumptions about who’s going to turn out to vote,” said Lake. “For example in 2014, millennials and people of color didn’t turn out to vote, so if you’re basing your turnout model on the past, then you’re going to be wrong.

“And people are trying, everybody’s trying to make adjustments for that, but how much you adjust is really a question.”

‘Vote’

People who bother to vote in midterm elections – or who have the time or means to do so – are older, whiter and more educated, as a group, than those who vote in presidential election years. That’s good for Republicans. When there is a Democrat in the White House, the GOP advantage grows a few points more.

But the November 2018 elections present a sharply different picture from the last two midterms. A Republican is in the White House, potentially muting the party’s likely voter advantage. Additionally, Trump is not a popular president, a factor that could discourage GOP turnout and goose Democrats.

Democrats have performed well in contests since Trump was elected, easily winning two gubernatorial elections in 2017 and outperforming the baseline in special elections that year. The Democratic advantage on the generic ballot sits at about 8%, according to polling averages.

But the failure of Democrats, particularly young people and people of color, to vote in past midterm elections remains a concern. Only 35.9% of eligible voters turned out for the 2014 midterms, a low point going all the way back to 1942. As a result, Republicans gained control of the Senate and, counting state legislatures and governorships, cemented their largest majority nationally going back almost 100 years.

In a relatively rare public address on Friday, Barack Obama encouraged Democrats to act on their frustration by voting.