BBC Question Time made a brief foray onto the national news agenda recently after panellist Laurence Fox accused an audience member of racism when she described him as a “white, privileged male” during a discussion about the media’s treatment of Meghan, Duchess of Sussex. The exchange set in motion a predictable chain of events: Fox began a tour of Britain’s TV and podcast studios, making a series of increasingly reactionary and attention-seeking statements. Across left-leaning social media, meanwhile, Fox was mocked with a mix of amusement, disdain and pity.

But focusing on the shallowness of Fox’s opinions elides the most important element of the Question Time spectacle: the fact that a significant chunk of the audience groaned as soon as the phrase “white privilege” was uttered. Fox was not the only person in that studio who was weary of contemporary antiracist discourse, and he wasn’t the only person willing to show it.

Indeed his Question Time performance is part of a trend for “anti-woke” celebrities, such as Piers Morgan (examples of his attention-grabbing antipathy to wokeness are too numerous to list here), the comedian Geoff Norcott, who complains about “lifestyle prefects”, and the Twitter parodist Andrew Doyle, who argues that “woke bullies” must be resisted.

It was therefore intriguing to see how many progressives regarded Fox’s outburst as a surprising event – as if the actor was a foreign object that suddenly crashed into our harmonious world of social liberalism. Not least because recent political developments suggest that there are millions of Laurence Foxes up and down the country, and that their views are mainstream. Note, for example, how David Walliams’ joke about Fox went down at the National Television Awards. Walliams implied that Fox would find himself friendless following his appearance on Question Time, presumably expecting laughter and not the chorus of “oohs” that came instead. Among liberals, Fox may be the object of mocking scorn – but his sudden notoriety is just one symptom of a growing “anti-woke” backlash that deserves closer examination.

The progressive tendency to regard “anti-woke” crusaders as aberrations is a hangover from the liberal consensus established in the late 90s. New Labour’s landslide victory in 1997 didn’t signal just a change in government, but an ostensible change in our nation’s culture. Exhausted and demoralised by the polarising Thatcher years, British people were apparently ready for a more liberal and tolerant era.

The new received wisdom dictated that women and LGBT+ people were equal (sort of), and racism was to be condemned (unless you were a Muslim). The reason liberals still believe this consensus holds is that the politics New Labour ushered in was so dominant and all-encompassing that almost every opinion that existed outside of it was dismissed as the view of cranks.

The most salient example of this is the Conservative party, which under the leadership of David Cameron recognised it would have to lean in to socially liberal values in order to gain a hearing. The culmination of this was that the Tories – historically the party of homophobic legislation – would eventually outflank New Labour by overseeing the introduction of equal marriage. In 2006, the Conservative and Blair critic Matthew Parris conceded in the Times: “Britain is a nicer place than when [Blair] entered Downing Street. Something tolerant, something amiable...has left its mark upon the country.”

In other words, social liberalism was not merely a popular point of view: it was the new normal. It was also fundamentally modernising. The idea of these newly founded values being contested would have seemed like time going backwards.

Now that same political consensus is collapsing across the world wherever it had been established. In its place is a new, young left that is more radical on issues of social liberalism, understanding that gender, sexuality and race are bound up with questions about power and privilege, and that these intersecting identities can produce significantly different life experiences. But as the tide of 90s social liberalism has ebbed, it has also revealed another group of people (primarily older, white homeowners and pensioners) who had never bought into the consensus in the first place, and are aggressively hostile to its newer, more radical iteration.

We all know a member of this demographic: alienated by the modern world and displeased by change, they are fond of complaining that “You can’t say anything any more!” – even as their opinions are widely reproduced in the nation’s print media. Perhaps they spent the 2000s retreating into the Daily Mail columns of Richard Littlejohn and his contemporaries, or simply feeling lost altogether. They are the people that have enabled Brexit and Donald Trump to succeed, and have since transformed themselves into the base of a potent political movement.

Having spent so long feeling silenced by the liberal consensus, people in this group have been given a new lease of life by the right’s new insurgents. Not only were they correct all along; they were actually victims, zealously persecuted by an oversensitive and censorious society. It is this righteous indignation that lends their antipathy to wokeness a defiant and almost celebratory quality. As a friend of mine puts it, we are living in “bigot Christmas”.

On the right, it is common to argue that the backlash to wokeness has arisen because identity politics has gone too far and rendered itself impenetrable to the majority. (This is an argument sometimes echoed in parts of the left and centre.) This argument is usually accompanied by outlandish examples of identity politics, such as a paper that suggested Greek yoghurt has been culturally appropriated, or a blogpost decrying “white veganism”.

There’s no doubt that these examples would indeed be incomprehensible to the majority of people. But the idea that ordinary people are being driven into the arms of authoritarianism because of an excitable article they read on the internet is facile – and any progressives adopting it should ask themselves why they are parroting arguments that are largely advanced by the far right. Indeed, if the political claims of people of colour and women really had gone “too far”, the distribution of power and wealth in the world would look very different.

Ultimately, Laurence Fox and others like him don’t want to hear about “white privilege” because it makes visible what has always been hidden – their power – and forces them to justify it. Power is nice, and liberating, and those who have it tend not to give it up without a fight.

Progressives need to wise up to the fact that they are losing this argument and decide what they are going to do in response. If they don’t, they may soon find that the future they always assumed was theirs is being made without them. Or as Florian Philippot, senior strategist to Marine Le Pen, tweeted after Donald Trump beat Hillary Clinton: “Their world is collapsing. Ours is being built.”

• Ellie Mae O’Hagan is a journalist and author