In January this year, I went on a Brexit trip to Dover. The main aim was to get a sense of what a no-deal exit from the EU would mean for the haulage trade, but while I was there I spent a long and cold afternoon wandering the seafront and town centre. Just near the white cliffs, on a jetty that seemed to form the last British land before the Channel, there was a dome tent, seemingly the property of someone who was homeless: an awful symbol of this country as a land of want and wilful neglect.

Later, in the centre of town, I met a woman who had been living on the streets for four years, having been repeatedly “sanctioned” and had her benefits stopped. The last time this had happened, she had been in hospital and failed to make it to the jobcentre, which caused her payments to be suspended for three months.

“You’ve got people here sleeping on the seafront, in huts, on the beach,” she said. “It’s disgusting.”

I was there, I told her, because of Brexit. “What is Brexit?” she asked. This was a sincere question. After a long pause, I told her that it meant our exit from the EU – which was proving to be long and complicated, meaning that politicians had little time to think of anything else, and the predicament of people like her was being forgotten.

Johnson’s antics have a flavour of pre-revolutionary France. To hell with poverty: here is Mr Optimism with a chicken

“OK, yeah,” she said. “So basically they don’t give a shit.” This seemed like fair comment; we chatted for 10 minutes more, I gave her money to buy some food, and we went our separate ways. Over the past three years, incidents such as these have punctuated all my reporting: small, sobering encounters with good people whose day-to-day experiences place them at the social edges, far away from the cacophony of chatter about customs unions, backstops and trade agreements. I have met single fathers in small West Country towns forced to get help from local churches; people in the West Midlands who, relocated by London councils after becoming homeless, now find themselves without friends, family or the most basic support networks; others laid low by cuts to social care, public transport or special needs education.

Some of them have had views on Brexit just as strong as anyone else’s. But others have understandably treated all the Westminster drama and Brussels negotiations as a completely irrelevant, borderline-absurd business that says nothing to them about their lives. Remember: nearly 30% of us did not vote at all in 2016. That sense of pained indifference lingers on.

Now, as Brexit reaches its supposed climax, another too often unreported world is all around us: 50,000 children being fed by local schools over the summer holidays; parents who are skipping meals to try to save their kids from hunger; weekly stories about the endless nightmares of universal credit. In even the most outwardly affluent towns, I now expect to see sleeping bags in the doorways of empty shops, and people asking for spare change. Meanwhile, we are encouraged to stare at a mad parade of Tory politicians convinced that this blighted, almost broken, country is in need of yet another unprecedented economic shock. Their new leader’s antics have a flavour of pre-revolutionary France. To hell with poverty: here is Mr Optimism with a chicken, or in his new “Prime Minister”-labelled coat, or arguing with his girlfriend about a spilled glass of wine.

Which brings us to one of 2019’s most overlooked questions: how can the mad politics of Brexit, the camp theatrics of the new prime minister and the constant sense of social breakdown possibly coexist? Worse still, how can the political party responsible seemingly be in with a very strong chance of winning an election reckoned to be just around the corner?

To some extent, Tory Brexiteers have pulled off the most devilish kind of trick, sowing discord and resentment via austerity, presenting Brexit as some kind of answer and reaping the rewards. In this reading, however irrational it may seem, much of the enduring support for leaving the European Union – up to and including the no-deal version – is a misplaced reaction to poverty, inequality and cuts.

I have been to plenty of places – Wigan, Merthyr Tydfil, Stoke-on-Trent – where this rings true, and people do talk about voting leave as a reaction to years of economic neglect. But it is not the whole story, foundering when it comes to why millions of people in comparatively affluent places voted leave. And herein, perhaps, lies something too often overlooked: that in many cases support for austerity and Brexit are one and the same thing – proof that, with the Tories’ encouragement, a whole swathe of public opinion has long since turned cruel and inward-looking, and it will take a hell of a shock to push it somewhere else.

Facebook Twitter Pinterest Nigel Farage supporters at a rally in Merthyr Tydfil in May 2019. Photograph: Sean Smith/The Guardian

In a recent book ostensibly focused on Jeremy Corbyn’s Labour party, but partly about recent British political history, the academics Matt Bolton and Frederick Harry Pitts explain the last decade in terms of “austerity populism”. Cuts, welfare crackdowns and the case for leave, they explain, were all sold to the public via the exclusion of supposedly unproductive undesirables: “scroungers” in the austerity narrative; “migrants” in the stories that swirled around the 2016 referendum.

Both traded on a nostalgic idea of national struggle, keeping calm and carrying on, and some strange, latent belief that the country was in need of a purgative spell of pain akin to an imaginary version of the second world war. In this vision, David Cameron’s election victory in 2015 and the leave side’s win a year later were watershed moments on the same national journey.

But if austerity populism has so far been politically successful, it also comes with obvious risks. Trumpeting the wonders of slashing services and kicking around the poor only works for as long as the majority of people are largely untouched by those things – which is why Johnson is now partly changing tack and pledging to spend money (although our nasty, broken benefits system and countless imperilled public services will surely remain untouched).

Boris Johnson says he’ll spend, but who will pay? Read more

By the same token, the romance of leaving the EU will only endure while its losers – sheep farmers, car industry workers, people who have come to the UK from central and eastern Europe – form a minority, and enough voters can still be persuaded that they will be winners in a Tory Brexit. Yet, however shambolic the opposition offered by Labour, the lived reality of no deal would surely risk tipping too many people into doubt and fear and away from the Conservatives, which is one reason why Johnson and his allies are in such an obvious hurry.

The emotional side of me would simply describe this all as a very English tragedy, centred on a mean-spiritedness that the woman I met in Dover would instantly recognise. And at least until the end of this long, overheated summer and the start of an autumn of nightmares, millions of us will carry on behaving much as we have done for the last decade: not just passing by on the other side, but dancing as we do it.

• John Harris is a Guardian columnist