In the coming days the blame for Britain's vote to leave the European Union will be distributed liberally among today's crop of politicians. David Cameron and George Osborne will, finally, be seen for the mediocre politicians that they are. Cameron has already announced without fanfare that he will step down as PM and a new Tory leader will be in place by October. With Osborne as equally tainted by Brexit, the smart money is on Boris Johnson to be the next Prime Minister.

Meanwhile, Labour leader Jeremy Corbyn will rightly be slaughtered by his MPs for being practically invisible for most of the referendum campaign. Even Alan Johnson, the best leader of the opposition Britain never had in the eyes of many, has singularly failed to ignite the passions of Labour's former heartlands in the north of England.

But taking a longer view, New Labour probably carries more of the blame than Labour's current leadership for Britain crashing out of the EU. Had the former demanded transitional controls on Eastern European migration way back in 2004, there's every chance that we would be waking up today to a resounding victory for Remain. It was, as the former home secretary Jack Straw recently admitted, a 'spectacular mistake' to throw open the doors in 2004, a 'well-intentioned policy we messed up'.

Labour expected only around 13,000 migrants a year to arrive in Britain; instead the figure was in the hundreds of thousands, hitting a record level of 333,000 just four weeks ago. In many ways the influx of migrants from the former eastern bloc countries had a tremendously positive effect. Migrants have done the jobs that Brits have been unwilling to do and they have contributed far more to the exchequer's coffers than they've taken out in return. Ultimately, they have been quietly paying for the pensions of those who've just voted to kick them out.

For all the economic benefits, immigration on this scale was incredibly unpopular, even among recent immigrants

But for all the economic benefits, immigration on this scale was incredibly unpopular, even among recent immigrants. Poll after poll told us as much. According to a recent British Social Attitudes Survey, 60% of those who came to Britain in the 1960s and 1970s said they wanted to see a cut in immigration. Meanwhile 39% of non-UK born white respondents earning £75,000 per annum reported preferences for 'a lot less' migration.

The former Tory leader Michael Howard ran his party's 2005 election campaign on a platform of reduced migration, so large and unexpected was the initial influx from eastern Europe. The party took out a full-page advert in the Sunday Telegraph calling for 'an annual limit on immigration and a quota for asylum seekers'. But it was too soon. There was certainly resentment back then.

Together with the Conservative election campaign, the far-right started to reappear in down-at-heel northern towns after almost 20 years in the doldrums. But Labour politicians promised to listen to voters' concerns about migration and it got them through. Shortly before the 2005 election The Sunday Times revealed that home secretary Charles Clarke planned to 'steal part of the Tories' immigration policy by announcing a new Australian-style points system for economic migrants'. And so for a time the public believed it and gave them the benefit of the doubt.

But the promises to 'listen', to 'get serious' and to 'respect people's concerns' sounded increasingly hollow as the years passed and European migration to Britain continued to soar to record levels. Those who banged the drum the loudest on immigration were often racists, thus it was assumed by well-meaning progressives than anyone who emitted even the mildest squeak of disquiet about immigration were, if not racist themselves, then happy to play the sordid politics of the 'dog whistle'.

Anyone who wishes to lazily ascribe racism to more than half the electorate is making the very mistake which got us into this sorry mess.

Nuance fell right out of the debate. Immigration was either a boon to the British economy or it was irreversibly changing the nature of the country at a speed which most people were decidedly uncomfortable with. Anyone who pointed out that it might be both was drowned out by the cries of 'racist' from one quarter and a pack of lies about migrants 'milking the benefit system' on the other.

When people were listened to on immigration, their fears were quietly put down to false consciousness. Their grumbles were, it was said in polite circles, code for something else: concerns about jobs, wages or the size of the mortgage. Jeremy Corbyn perhaps epitomised this sense of detachment from reality better than anyone. Even following the referendum result he has persisted in saying that the Leave victory was down to jobs, housing and the same old material things that cod-Marxists like Corbyn believe can explain everything.

There is of course some truth to materialist explanations, but they don't give the whole picture. Hostility to immigration – and by extension hostility to Europe – is driven by cultural concerns as much as by economic worries. That's certainly what the University of Oxford's Migration Observatory has been saying in recent years. It has pointed out on a number of occasions that cultural concerns better explain negative attitudes towards migration than a person's economic position. In essence it is about whether England feels like England. And that is no more the England of Enoch Powell or the English Defence League than it is the England of George Orwell, who wrote of 'something distinctive and recognisable in English civilisation. It is a culture as individual as that of Spain'.

In urging voters to 'take back control', the Leave campaign tapped into this in a way that the Remain camp, with their statisticians and endless parade of captains of industry, was unable to. Yes, racism played a part; but anyone who wishes to lazily ascribe racism to more than half the electorate is making the very mistake which got us into this sorry mess.

The tragedy is that Brexit is unlikely to reduce immigration, nor improve the economic prospects of resentful working class voters...it won't be the Nigel Farage's and Boris Johnson's of the world who will feel the pinch as the British economy takes a hammering.

The tragedy of course is that Brexit is unlikely to reduce immigration, nor improve the economic prospects of resentful working class voters trapped in economic turpitude in the grimmest corners of England's north. At a time when placing any political bet is a high risk endeavour, you can bet the house on the fact that it won't be the Nigel Farages and Boris Johnsons of the world who will feel the pinch as the British economy takes a hammering.

Beyond Britain's shores, the rise of the far-right now looms ominously over the European continent like a fearsome rain cloud. Fascism is the small man writ large, and the small man (and woman) is in the ascendancy. France's National Front leader Marine Le Pen has said that the French must also have the right to choose. Meanwhile Dutch anti-Islam politician Gert Wilders and Italy's far-right Northern League have said much the same thing.

We are witnessing nothing less than the creeping break-up of Europe. It will go out with a whimper rather than a bang, and it was set in motion a decade ago by Labour politicians who saw the English working class as a superfluous force who had nowhere else electorally to go. They pushed and pushed and pushed them and today, finally, the great unwanted have pushed back. The salt of the earth were treated as the scum of the earth and, unsurprisingly, they wouldn't stand for it. The dark consequences will be felt for generations to come.