Not long after the referendum in 2016, some neighbours suddenly erected a big wooden fence around their front garden. Well, it seemed sudden. Our postman jokingly called it “the Brexit wall”, and although the two events may have been entirely unconnected, the fence took on a symbolism it was probably never intended to have.

Whether it was there for reasons of privacy, security, or simply to have something for the ivy to grow on, we’ll never know. I don’t like to ask. But the fence reinforced my belief that community life has never been, and never will be, as straightforward as some dewy-eyed nostalgics would like to present it. That thought recurred this week, as the country staggers towards a general election. Whenever it does finally take place, both the Tories and Labour will make a pitch – in the Midlands and the north of England particularly – on the presumption that the leave vote represented a desire to take things back to how they supposedly once were.

The fence incident, and the sense of privacy and desire for individual space it seemed to suggest, reminds me of many of the interviewees featured in the historian Jon Lawrence’s new book, Me, Me, Me? The Search for Community in Postwar England. In it, Lawrence rescues dozens of voices from dusty transcripts buried in university libraries to build a picture of how British people have felt about their lives and the places they live in over many decades.

When government fails and basic needs go unmet, people turn in on themselves and try to look after their own as best they can

These are not vox pops. In successive studies people were interviewed at length, over many months, at work and at home, and encouraged to put across their side of the extraordinarily complex story of postwar society. What they reveal, unsurprisingly, is that people hate being told what to do: whether by family members, neighbours, bosses, colleagues or politicians.

But at the same time they do want to feel supported. When government fails and basic needs go unmet, people turn in on themselves and try to look after their own as best they can. The idea that community is bolstered by character-building hardship is a myth. An unemployed Sheppey couple interviewed by the sociologist Ray Pahl in 1982 told him that when it was a struggle to maintain family life, it made you “sort of more narrow”. In Lawrence’s words, “the outside world ceased to matter when they were struggling to survive”.

Lawrence reminds us of what we should never forget: that to experience class at the “wrong” end of the scale – the economically difficult, culturally denigrated, politically marginalised end – is a constant struggle against visible and invisible forces. People’s ambitions for themselves, their families and their neighbourhoods are vulnerable to attack from a combination of wilful inaction and deliberate destructiveness from government.

It was undoubtedly resentment at such malign neglect over past decades that drove much of the Brexit vote in 2016, among traditionally Labour working-class communities in the north of England and the Midlands.

But the result has been a series of tired popular narratives about the role of place in forming our identity, the most common being the idea that certain places have been “left behind” – a term that conjures up a vision of areas filled with people who are bewildered and marooned in a changing world. Those places haven’t been left behind, but deliberately held back. The people in them are desperate to vote for a party that is fundamentally optimistic about our ability to cope with change and knows what it has to do to make it possible. Labour’s 2017 election manifesto did this. It boosted the turnout and won the party over 40% of the vote. That will not stop the Conservatives attempting to mount a land-grab of supposedly “left-behind” Brexit voters. (It was no accident that Boris Johnson chose Wakefield as the backdrop for his rambling speech yesterday.)

A recent report by the right-leaning thinktank Onward entreated Tories to place more emphasis on “the politics of belonging”, suggesting that “voters do not want more autonomy, choice and mobility. They want a government that … protects them, their families and British businesses from the modern world.”

Do they mean the same modern world in which the British use smartphones, social media and cashless payments more than anyone else? The problem is not modernity, or nostalgia for an idealised close-knit “community”. People desperately want the feeling of autonomy, but are intelligent enough to recognise that autonomy isn’t possible without an underpinning of material security. And they are not going to get that under the Tories, whatever guise the party takes under the shape-shifting Johnson, and whatever pre-election bungs are cynically tossed their way by Sajid Javid.

The Labour party too – and for a longer period of time – has been misreading the politics of community. When Ed Miliband was leader at the start of this decade he convened a lot of workshops and roundtables with advisers and MPs, asking: “What should Labour conserve?” In 2011 was invited to one of them, and I once got into a barney with two MPs who insisted that Miliband’s emphasis on universal benefits would lose Labour “the white working class” for ever if they felt that migrants and recent arrivals in communities were given equal treatment to longstanding residents. “It’s about community, and social cohesion, and values,” they blustered. What right had I to challenge the idea that the white working class – as they chose to define it – were right about all things at all times?

Their approach was to paint Labour’s problem as cultural – an inability to understand “how the working class thinks” – rather than one of policy. Labour spent decades accepting the policies that left deindustrialised England battered, bruised and angry. In the 1990s and 2000s it believed that if it spent enough money on welfare and education it would cover the cracks. When the cracks widened and working-class voters began to desert the party, they went for the “culture” explanation.

How the left lost the fight for Englishness | Suzanne Moore Read more

Successful community life is not about who chats to whom on the doorstep or about mystical working-class values. It is about whether the essential needs of everyone in that community are met to the extent that they can look beyond that day’s survival and outwards towards others. In Lawrence’s wise words, “the task for politicians is to imagine the conditions which will enable people most fully to reconcile self and society in their daily lives”.

So when the Conservative party sends their representatives to court the working-class vote, they should be asked a few questions. Have they ever tried to apply for benefits to which they are entitled, or sought help from Citizens Advice? Have their local branch library and children’s centre been closed? Have they ever tried to get a bus on a Sunday, or after 8pm, outside London? Have they noticed that children no longer play out on the street? Have they ever peeled “Free Tommy Robinson” stickers off the bus shelter and the playground gates? What did they feel on visiting any city centre and seeing the lines of destitute people trying to keep their tents pegged to the pavement and their sleeping bags clean?

The very idea of a culture war, invoking supposed working-class values, is empty. What we are really talking about is quality of life. The problem of our times and our communities is that a full, rich life, free of avoidable stresses, in which the right to stay and the right to move are equally taken for granted, now feels so hard to achieve for so many people.

• Lynsey Hanley is a writer and the author of Estates: an Intimate History and Respectable: Crossing the Class Divide