Towards the end of the 1990s some Labour insiders began to notice that economically disaffected working-class voters were drifting into political apathy. After the high of Tony Blair’s landslide election in 1997, Labour’s vote dropped by 3 million in 2001 (even though it won another landslide) and by a further million in 2005. A small number of these disaffected voters were attracted towards the BNP, but this party’s racist connections were a turn-off for many.

But from 2010, as the BNP began to fall apart, these voters were targeted by Nigel Farage. Ukip had previously been a party of the Tory shires but began to emerge as a serious force in many traditional Labour areas. Ukip began to win over Britain’s left-behind voters who felt intensely anxious over immigration, saw the cause of it as Britain’s EU membership, and loathed a Westminster elite that appeared uninterested in their concerns.

The EU referendum marks the latest episode in this long-running story about Labour’s fragmenting coalition and its deteriorating electoral fortunes. As the vast majority of its MPs have spent weeks calling for voters to back remain, many voters in these Labour seats will have instead given strong support to Brexit.

The rise of Ukip was a key reason why Labour was unable to return to Downing Street in last year’s election. The root of Ukip’s appeal was misdiagnosed by Labour MPs, who refused to accept that it was as much about identity as economics – an abstract feeling that the EU, immigration and rapid social change threatened a cherished identity, community and set of values.

Ukip, which drew most of its votes from older, working-class and self-employed Britons, emerged from the 2015 general election as the main opposition in 120 seats, 44 of which have Labour MPs. Nationally, Ukip averaged nearly 13%. But in Labour’s traditionally safe northern heartlands this figure jumped to 19%.

The social divides that explain Ukip’s advance relate to social class, generation and geography. And they cut right across the Labour electorate. Britain’s referendum positioned a professional, younger, university-educated and urban middle-class against an economically struggling, older, poorly educated and left-behind section of society. One group feels broadly at ease with EU membership, free movement and rapid social change; the other feels fundamentally under threat from all these things.

Labour is dependent on both these groups. And it is the latter, the left behind, who would have watched Jeremy Corbyn making the case for free movement in the past few weeks with deep unease.

Throughout the referendum campaign, between one-fifth and one-third of Labour supporters said they wanted to leave the EU; and the electorate in 59% of all Labour seats were predicted to have voted for Brexit yesterday. The strongest levels of support were in the places Labour and Corbyn are beginning to struggle the most – northern, left-behind and traditional Labour seats, such as Blackpool South, Dudley North, Walsall North, Rotherham, Doncaster North, and West Bromwich West. Indeed we already have evidence of how Labour’s hold over voters in these more traditional areas has been weakening.

By contrast, other Labour seats were predicted to deliver some of the strongest support for the EU – such as the leafy London seats of Hornsey and Wood Green or Hampstead and Kilburn, the young and socially mobile Bristol West, Cambridge and Manchester Withington.

Some senior figures, such as John McDonnell and Chuka Umunna, will argue that by making the case for remain they are defending and advancing their progressive values, and they are right to do so. But such values differ fundamentally from those cherished by the traditional working class, whose concepts of community, solidarity and belonging are more exclusive and tribal than the internationalism of European federalists.

Some will also argue that Labour has failed to deliver a compelling argument about the merits of EU membership. During the campaign I was told by both trade unionists and Labour MPs how, in their view, the new leadership was not doing enough. Indeed, one in two Labour voters told YouGov they were unsure where Labour – the party of Tony Blair and Peter Mandelson, who wanted to lead the country into the single currency – stood on the EU.

But it goes deeper than this. Euroscepticism is a complex problem for Labour, but immigration scepticism is more widespread among Labour’s traditional voters than anti-EU sentiment. Nearly two-thirds of Labour supporters say they are unhappy about how immigration is being managed.

Not all of these voters view Brexit as the answer, but 38% feel the government should have total control over who comes into Britain, and 30% feel Britain should stop EU citizens coming into the country to live and work, even if that restricts our access to the single market.

Corbyn has so far shown little understanding of what is driving this identity angst. He has said little that would resonate among those Labour-to-Ukip defectors. And there is no doubt that these tensions hold the potential to pull Labour in different directions and make a return to power virtually impossible, certainly in 2020 and perhaps beyond. It is not yet clear how Labour can reconcile this deep divide, but there is little doubt that this will be its biggest challenge for decades.