“It’s a shithole,” she said. “It’s supposed to be Theresa May’s area, and, I’m not being funny, but would you want to live here?”

I’ve seen worse places, I said.

“Rates went up, shops closed,” she continued. “And now all we’ve got is coffee places and phone shops.”

A pause. “My grandchildren go to school, and all they’ve got is budget cuts, budget cuts, budget cuts. We’ve got Crossrail coming through here, and all it’s done is put up property prices and made the poor man poorer.

“I’m sorry – I could go on for hours,” she told me. “I used to think this was heaven. It’s just so rough. And I know she’s got Brexit and all that, but this is her constituency. You think she’d look after it.”

I was in Maidenhead, Berkshire, on a sunny Monday, making a video for the Guardian’s Anywhere But Westminster series. What I could see was hardly a wasteland, but I understood what this woman meant, and why it was echoed time and again by other people I spoke to, both young and old. The high street was smattered with empty premises. there was a familiar-looking array of discount shops and vape outlets. The chain of hairdressers Toni & Guy was missing most of the letters from its sign, which now said “Ton”; next door was a former branch of the Royal Bank of Scotland, long vacated and gathering dust.

There are plans to revive Maidenhead’s town centre, focused on a development called The Landing, which will contain office space, and hundreds of new apartments. On the face of it, it might help to ease the problems of providing homes in an area where 83% of the local land is defined as green belt. But to many people I spoke to, this new arrival threatens to compound one of Maidenhead’s problems: the fact that it increasingly exists to serve the needs of people priced out of the capital, who have no connection to a town they reluctantly live in, and who tend to spend their money elsewhere. Besides, even if this project comes good, it will not gloss over what will presumably still surround it: a weird mixture of affluence and decline, which feels as if it embodies an overlooked malaise that existed well before the 2016 referendum.

Twelve miles up the A404 in High Wycombe, the picture was even starker: housing developments going up at speed around a town centre in which any civic pride seemed to have all but evaporated. There were as many vacant shops as I have seen in places that are outwardly much more deprived. The local MP is Steve Baker, the self-styled “Brexit hard man”, who seems happy to chase his revolutionary dreams and immerse himself in Tory infighting, even as his political backyard decays. Few people I spoke to recognised his name: rather than talk about indicative votes and emergency summits, they were much keener to dwell on their town’s predicament. It was symbolised, many said, by the fact that even a new(ish)shopping centre called Eden – Eden! – was now spotted with vacant units.

Just as I grew up in an England where the optimism of the 1960s – all tower blocks, shopping parades and modernist multistorey car parks – was falling into disrepair, so something comparable seems to have happened over the last decade. From the 1990s through to the crash of 2008, affluent England was sold a dream of supermarkets, bowling alleys, multiplex cinemas and aspirational coffee outlets – all of which pointed the way to a comfortable, pleasure-filled 21st century. That dream is now fading, and nothing has come along to replace it. The tables in the chain cafes are sticky and chipped; either side of them are either charity shops or empty gaps. The commuter trains are overpriced, late and crowded; the roads are full of potholes. The best anyone can hope for, perhaps, is the chance to disappear into their starter home, and momentarily forget about what sits outside.

If the agonies of Brexit and the deep crisis that currently besets English Conservatism are symptoms of deeper problems, here they are. Maybe the die was cast 40 years ago, when leading Tories embraced the credo of market fundamentalism and began to forget about civic pride and active government. Perhaps the takeover of Conservative politics by the London clique headed by David Cameron and George Osborne made the problem even worse, not least because of the austerity that has ensured that even Buckinghamshire is now facing dire financial problems – four district councils have just been folded into a new unitary county council in a drastic attempt to save money, the ruling Tories recently announced £20m of budget cuts spread over four years, and there are rising fears about social care, special needs education and plans to close no less than 19 children’s centres.

With Brexit eating the Tories up, any notion of how to address such modern issues as the state of local public services, the migration of consumer spending online or our awful public transport now seems to be beyond their collective grasp; by way of ideas, all they have to cling on to is the old ideal of the property-owning democracy, which has long since shut out people under 40, and thus proved to be not very democratic at all. It happened to be the 2016 referendum that came along and tipped them into disarray and division, but let’s face it: they were due a crisis anyway. As more clued-up Conservatives surely realise, the absence of any serious Tory thinking about the nitty-gritty of our national future will sooner or later have huge political consequences for their party; in the sense that Brexit is partly a misplaced expression of people’s seething anger about Britain’s current condition, it arguably already has.

Windsor and Maidenhead voted to remain by 54% to 46%; in the local government district of Wycombe, remain also won, with 52% (Baker, who had said he would resign if remain won, said he was “disappointed” but “also surprised”). In both towns, many people I spoke to on either side of the Brexit divide were weary and exasperated by the parliamentary pantomime, and so keen for it all to quieten down that many of them seemed open to settling for whatever compromise the politicians could come up with. What they were most comfortable talking about was what might happen to their immediate surroundings – and, by extension, the future of the country as seen from the perspective of everyday life.

There is a question I ask most of the people I talk to about Brexit and everything bundled up in it: how do you feel about the future? I have heard people answer it with anxiety and anger in lots of places where disaffection and resentment runs deep, and goes back decades – the post-industrial north of England, the south Wales valleys, the most populous parts of urban Scotland. But if even the Thames valley shows signs of dread and dysfunction, where exactly are we? And as the current disaster grows greater by the day, where will we end up?

• John Harris is a Guardian columnist