Twenty years after New Labour’s triumphant electoral victory, Tony Blair and Gordon Brown are still squabbling. But far more important than the competing accounts of when the former agreed to make way for the latter is understanding why the promise of New Labour turned out to be false.

We all remember Tony sipping champagne with Noel. A wall of flowers for a people’s princess. Damien’s shark in the Royal Academy, just a few rooms away from Tracey’s tent. Geri in her Union Jack, proclaiming the rise of girl power. Doreen Lawrence demanding an inquiry for her murdered son. In 1997, these were seen as harbingers of a fairer, more open and more modern Britain. Today, after a decade of crisis, protest, riots, racism and referendums, we know only too well that New Labour and the liberal culture that enveloped it did not create a “new Britain”.

The English establishment has always had a knack for spotting a good (and subversive) idea – before quickly making it their own. That’s why there hasn’t been a revolution in England since 1688. From the Great Reform Act of 1832 to the construction of the welfare state, grassroots agitators, parliamentary radicals and news of revolution abroad have forced the establishment – both recalcitrant reactionaries and liberals willing to compromise – to absorb a few radical ideas and save itself. In doing so, it strengthened the existing order further.

In 1997 New Labour took this establishment penchant for pursuing marginal reform that reinforces the status quo and developed it into the operating principle of a whole political culture. It defined the historical moment. In his new office at the Treasury, its walls painted brilliant white and its floors cheaply carpeted, Gordon Brown oversaw an ambitious expansion of spending on schools, healthcare and his new Sure Start scheme.

And so the Labour government made marginal but valuable improvements: the neoliberal juggernaut slightly decelerated. But there was no attempt to stop, let alone reverse, it. This would have meant reconstructing the economy and the state on fundamentally fairer terms. Inequality rose more slowly under New Labour than it had during 18 years of Tory rule. Nonetheless, it rose substantially.

Brown’s plan turned out to be not only unambitious, but counterproductive. He funded his expansion of the welfare state using revenues generated by continuing the deregulation of the City, which allowed financial markets to take new risks and make enormous profits that he expected would swell the Treasury’s coffers. But since the 2007 financial crisis, austerity and stagnation have swept away New Labour’s painfully incremental improvements.

Something similar happened when New Labour attached itself to the Stephen Lawrence campaign, and then made only the slightest dent in the racism institutionalised in the state and society. Jack Straw and Tony Blair emphatically endorsed the finding of a judicial inquiry that the failed Metropolitan police investigation into the Lawrence’s racist murder had been “marred by … institutional racism”. This contributed towards a progressive myth of a more tolerant future. Optimistic liberals pointed to the increasingly taboo nature of racist slurs, and the wide acceptance of mixed-race families. But two decades later there has been little change to the racialised imbalance in who gets stopped and searched, and in who suffers police violence. The class divide between white and ethnic-minority communities remains deep and wide. The slow diminishing of personal prejudice not only did nothing to tackle institutionalised racism: it provided cover for inaction.

This dynamic – powerful systems and rigid inequalities being strengthened by changes and reforms that might have been expected to diminish them – reached beyond Whitehall and into our pop culture. The year that Blair moved into Downing Street, the Spice Girls became the biggest band in the world. They did something that, for a totally artificial, middle-of-the-mainstream pop band was exceptional and extraordinary: they introduced their young fans to explicitly ideological ideas of independence and solidarity. “We have come up against a few guys who expect five bimbos,” Geri Halliwell told a reporter. “It just means you’ve just got to shout a bit louder to get your way. We’re freshening up feminism for the 90s.”

‘Too often, however, the Spice Girls’ marketing taught their fans to value conventional good looks and fashion.’ Photograph: Nils Jorgensen/Rex/Shutterstock

Too often, however, the Spice Girls’ marketing taught their fans to value conventional good looks and fashion. Girl power was, as pointed out by the US journalist Rachel Giese, “feminism-lite”; but it was also a marketing ploy designed to turn little girls into demanding and sexualised consumers. Fortune magazine doled out cynical marketing advice: “If you want to sell to the girl power crowd, you have to pretend that they’re running things, that they’re in charge.” This “girl power” wasn’t simply an accessible version of feminism; it was its free-market antithesis – a commercial system that used against women not just women’s lack of privilege, but also their political desire for equality. Geri Halliwell was not in on the con: just as Brown wanted to spend as much as he could on Sure Start, she thought the Spice Girls could make a small difference.

Nonetheless, today we are disappointed that the optimistic vision of 1997 failed to come about. Its compromises and appropriations, presented as acts of grand reform, have paved the way for our current upheaval. People are angry about the cost of living, racism, misogyny, the distance between themselves and centres of power.

Under New Labour the tension between the promise of reform and the reality of the status quo became an unsustainable contradiction. Blair insisted, from the beginning, that what a society worn down by Thatcherite market globalisation needed was more of the same. But he justified this by appropriating the rhetoric of both social democracy and national greatness. He announced, in his conference speech a year after entering office, “a new agenda: economies that compete on knowledge, on the creative power of the many, not the few; societies based on inclusion not division”. He justified Britain’s international role on the basis that “I am a patriot. I want Britain strong.””

Blair built support for the pro-globalisation policies that were eroding the welfare state by promoting nationalist feelings of regret for that erosion, creating a devastating sense of disruption for Britain’s “left-behinds”. His nationalist rhetoric, along with his anti-asylum policies, helped to ensure that when the backlash came, it did so in the isolationist and xenophobic form of Brexit and anti-migrant feeling.

When Brexiters – I’m looking at you, Boris Johnson – talk of “letting the British lion roar” and an extra £350m a week for the NHS, are they also playing the establishment trick of branding stasis as change? They should consider how they will be remembered if Brexit also delivers more of the same – if it reinforces the conditions that motivated many leave supporters: de-industrialisation, precarious employment and insecure housing, and the rolling back of the welfare state.

If that’s the kind of Brexit we get, then history will condemn the current government and its promises of transformative change even more severely than it has judged the happy, smiling and optimistic administration that came to power 20 years ago.

• Richard Power Sayeed is the author of 1997: The Future That Never Happened