Here he comes, wielding the delayed memoirs that will finally go on sale on Thursday, about to star in the two-part BBC documentary that premieres the same day, and committed to the bare minimum of public appearances. In the midst of such a huge national mess, David Cameron seems to be fulfilling a role that will be familiar to any devotee of country music or TV soap opera: that of the estranged, emotionally constipated husband, who left behind the domestic carnage he created and then hesitantly returned, lost as to what to say but desperate to somehow make amends.

The media has a habit of reducing complex and confounding questions about recent history to a single supposedly momentous event or decision, and so it is proving this time. With Tony Blair and his legacy, everything still supposedly centres on Iraq; in Cameron’s case, 95% of the weekend’s headlines were focused on what he now says about the promise, delivery and conduct of the referendum of June 2016. “I should have done more to mix criticisms of the EU with talking about its very real achievements,” he writes, a sentence that is surely a shoo-in for the understatement of the decade. The two big leave campaigns, he says, were “a cauldron of toxicity”. Boris Johnson backed his side – no, really – “to benefit his political career”, and Michael Gove was transformed into “a foam-flecked Faragist”. His anger and contrition, however, don’t encompass the vote itself: putting our membership of the EU to the public, he reaffirms, was “necessary and, I believe, ultimately inevitable”.

But fixating on the decision to hold the referendum and his woeful politicking with the EU ignores the nitty-gritty of Cameron’s record, and thereby lets him off the hook. Brexit, after all, is symptomatic of things that run much deeper than our relationship with Europe. And on this score, Cameron and his former colleagues have huge questions to answer, if only someone would ask them. In the wake of the crash of 2007-08, they took control of a country that was already starting to show signs of fury, division and seething resentment. Their biggest sin, it seems to me, was to encourage and accelerate those things, to the point that they were probably always going to explode in some awful drama of national self-harm.

As I was getting ready to write this piece, I re-read an essay about Cameron I wrote for the New Statesman in January 2006. It gave off the scent of much more innocent times, two and a half years before the fall of Lehman Brothers, when Blair was the prime minister, and we were still happily stuck in what we might think of as The Long 1990s. Cameron was keen on cycling to work and talking about “social justice, the dearth of female MPs, the gender pay gap [and] organic food”. He was also frantically praising the voluntary sector, and laying the ground for the vague mess of ideas he called the Big Society.

Senior politicians are usually strange creatures, with an amazing capacity for holding apparently contradictory impulses; in the moment, Cameron probably believed that what he was fumbling towards was not just human and kind, but suited to a world in which people can self-organise as never before. But it was also clear that woven into all this stuff was a very familiar Tory antipathy towards what remained of the postwar settlement. In his interview with Cameron at about the same time, the Observer’s Andrew Rawnsley noted “his attachment to that Thatcherite phrase ‘rolling back the state’”.

Brexit is partly the fault of irresponsible politicians who were not actually Brexiters

We all know what happened four years later. Mere days before the election of 2010, Cameron said that any Tory minister considering “frontline reductions” would be sent “straight back to their department to go away and think again”. He then commenced the decade of fiscal savagery that has left some of the most fundamental parts of the public realm hanging on for dear life: “an economic rescue job”, he now says, with his seemingly unshakable doublethink summed up in the claim that “the last thing I wanted to do was to make any cuts to anything”. The triteness is excruciating. Quite apart from the fate of adult social care, children’s centres and other vital services, Britain is now a country in which austerity is part of the everyday ambience, all shut-down pools and libraries, broken-down parks, and once-a-day buses. These things inflamed the sense of neglect that played a key role in the summer of 2016: credible academic research shows that pre-referendum support for Ukip correlated strongly with local austerity.

There is a belief around at the moment, encouraged by many of the 21 Conservative MPs recently expelled from the party by Johnson, that it is only now that their party has become unprecedentedly nasty and irresponsible. But in 2013, I can clearly recall seeing huge hoardings at the Conservative conference in Manchester with such crude slogans as “Welfare capped”, “Immigration down”, and the dog-whistling formulation, “for hardworking people”. Cameron’s big speech reduced Labour’s time in power to “millions coming here from overseas while millions of British people were left on welfare”. He also assured his audience that “we must act on immigration directly … and we are. Capping immigration. Clamping down on the bogus colleges … If you are not entitled to our free National Health Service, you should pay for it. If you have no right to be here, you cannot rent a flat or a house … When you are a foreign prisoner fighting deportation, you should pay your own legal bills. If you appeal, you must do it from your own country, after you’ve been deported, not from here.”

No one should now let him get away with styling himself as the failed pioneer of a softer, more sympathetic kind of politics. Whether or not he was privately uncomfortable with them, these words alone confirm not just that Cameron looked like an enthusiastic participant in the hostile environment doctrine that led to the Windrush scandal, but that Faragism had already infected the Tory soul.

Two years later, having fused all this stuff with the crabby English nationalism awakened by all those warnings about Labour teaming up with the SNP, the Tories won an outright parliamentary majority, but that was only this agenda’s first victory. It doesn’t really matter that Cameron himself limply campaigned to stay in the EU and then shamefully departed the scene: if anyone agreed with the essential thrust of what he had been doing for the previous six years, how were they ever going to end up voting to remain?

Politicians’ autobiographies are invariably tedious, truth-eluding things, often destined to be piled up in discount shops. Historians and political anoraks will take what they can from this latest example, but it will not shine much light on connections that are too often ignored. Brexit is partly the fault of irresponsible politicians who were not actually Brexiteers; Johnson’s nasty opportunism is not that different from the approach taken by Cameron and George Osborne, two people whose role in Britain’s current disaster is still under-appreciated. Whatever alibis and excuses are given an airing this week, these things ought not to be forgotten.

• John Harris is a Guardian columnist