Do people vote by values or interests? This question harks back to the dawn of Labour politics. Whether it was Disraeli’s ‘angels in the marble’ or Thatcher’s aspirational working classes, the left has always struggled to understand why people vote “against their interests”.

With Middlesbrough turning blue and Kensington red, the June election re-ignites this debate. It has prompted some to predict a full realignment of the right-left spectrum, and others to suggest the US culture wars have arrived on British soil.

We analysed polling before and after 2017’s extraordinary election campaign to help understand this. This was based on two 2,000-sample surveys by our partners CDSM – the first before the Labour surge, the second after the result. It was built around Values Modes, the psychographic tool we used in the Cruddas Review (which examined the reasons for the Labour defeat at the 2015 general election), as well as in projects for IPPR and the TUC.

Values Modes establishes core motivations. It splits the population into three groups: altruistic Pioneers (driven by ethics and inner-fulfilment), aspirational Prospectors (driven by image and individual success) and socially conservative Settlers (driven by safety and anxiety about change). Each group makes up roughly a third of the population. The tool has been developed through opinion research since 1973, and we find that it works better than metrics such as social grade (A, B, C1, C2, D, E, etc), which can overstate socio-economics and underplay how people feel.

We wanted to understand how the population is changing, and to look at how the Cruddas Review findings had evolved over a tumultuous two years.

The initial pre-surge poll (fieldwork late April/early May) showed a Tory lead of 15 points. Theresa May’s party led in all three values groups. The lead among socially conservative Settlers was especially extreme, at nearly 30 points. But even among Pioneers – who are culturally left-leaning – the Conservatives were ahead.

Labour’s exceptional campaign saw Corbyn claw back significant numbers from all three groupings. The final result, however, still shows an electorate divided by values. Pioneers, who tend to be educated, affluent and urbane, ultimately rejected May’s social conservativism. Supplemented by the more liberal of the Prospectors and some previously apathetic Settlers, they enabled a major upset. Historically, class differentiations meant that all three parties had a balance of all three values groupings. But Labour is now the definitive Pioneer party, and the Conservatives the principal Settler one.

Labour late April Tories late April Labour June 8 Tory June 8 Pioneers 31% 38% 48% 31% Prospectors 33% 48% 43% 37% Settlers 18% 57% 35% 49%

This gets interesting if we look at Labour’s values coalition over time. The heat maps below demonstrate which values groups Labour’s vote came from over the past four elections. (Warmer colours show the strongest concentrations of support.)

As we can see, Labour’s appeal became steadily more concentrated during the 2005-2015 period. The party went from pleasing all the people some of the time – in the twilight of the Blair era – to pleasing some of the people all the time under Ed Miliband. May’s assumption, when she called the 2017 election, was presumably that Corbyn pleased none of the people any of the time.

This proved false, with Labour building beyond the Pioneer cohort more successfully than Miliband had. In particular, Corbyn did better than expected among the acquisitive, individualistic Prospector segment. This is a skittish tribe, which tends to desert Labour in its hour of need, often falling into the non-voter, swing voter or ‘shy Tory’ categories (as they did in 2015). This time around, they were perhaps motivated by Labour’s retail policies and sense of optimism – especially when faced with a Conservative Party which didn’t discuss the economy or indeed offer much at all for the aspirational pragmatist. The challenge for Labour will be to keep hold of them as circumstances change.

Another interesting aspect of the picture is Labour’s purplest patch in 2017, which was a notable island of hostility to the party in 2005. One wonders if this is the place on the values map where anger about Iraq was most pronounced.

By looking at the equivalent Conservative heat map for the last four elections, meanwhile, we see good news from Labour’s perspective, in that May’s party also seems to be shrinking its values appeal. The Tory centre of gravity has moved away from Prospectors and softer Pioneers, whom Cameron’s detoxification project wooed. They have returned to a Settler core – a contracted version of the Michael Howard values coalition. Moreover, this group has become steadily smaller over time: 56 per cent of the population were Settlers in the early 1970s; it’s now closer to 30 per cent.

So, our Values Modes research implies that we’re seeing a distillation – or a narrowing, depending on how you look at it – in the values reach of both parties.

This comes despite the large vote shares enjoyed by both. And it feels even more pronounced when you examine the Brexit vote. As the heatmaps below show, the Remain vote looks remarkably like Labour’s 2017 vote, and the Leave vote rather like the current Tory base.

This does indeed suggest a cultural disconnect based on values over interests. It’s perhaps not a million miles from the distinctions others have drawn, between ‘Somewheres’ and ‘Anywheres’, between ‘Nationalists’ and ‘Globalists’, and between ‘open and closed’.

Moreover, it shows what a vital issue the EU – which acts as a proxy for these questions – is for both parties. This is especially true given Labour’s position doesn’t seem to match that of its members. Electorally speaking, Brexit may have been a surprising ‘sleeper issue’ in 2017. But looking at the above heat maps, you can’t help feeling it’ll have woken up by the time the country next goes to the polls.

David Evans is the director of the Campaign Company and a former assistant general secretary of the Labour party.