It could be argued that the September 2016 speech in which Hillary Clinton said “you could put half of Trump’s supporters into what I call the basket of deplorables” was the moment she lost the White House. The phrase was rapidly deployed to define her – and liberals more generally – as out-of-touch elitists who dismissed voters as “racist, sexist, homophobic, xenophobic, Islamophobic – you name it,” in Clinton’s own phrase.

What was easy to miss was the second half of the quote, which posed a more difficult question for liberals: what about the “other basket” of Trump voters – “people who feel that the government has let them down, the economy has let them down, nobody cares about them, nobody worries about what happens to their lives and their futures; and they’re just desperate for change. It doesn’t really even matter where it comes from.”

Any suggestion that a vote for Brexit or Johnson might express darker motivations is always met with a gloating snarl

Yet the frame was frozen before the second part of Clinton’s statement. The word “deplorable” hung over her campaign like a toxic cloud. She suggested, in her book What Happened, that it contributed to her defeat.

A similar difficulty has haunted liberal discussions of voters who chose leave in the 2016 EU referendum. It became impossible to talk honestly about the complicated motivations for that vote. If its words were too strong – if anyone suggested that racism or xenophobia played a part – this was seized on as classic smug, elite contempt. “A slur on 17.4 million people”, Mark Francois called it, when Will Self said one didn’t “have to be a racist or an antisemite to vote for Brexit. It’s just that every racist and antisemite in the country did.”

Remainers and Labour supporters were constantly accused of losing the argument because they were condescending to voters. Any suggestion that a vote for Brexit or Boris Johnson might express darker motivations than simply admiring the Tory manifesto is always met with a gloating snarl: no wonder you lost.

So the real business of understanding what kind of resentments may motivate some of these votes – and, more seriously, how politicians and the tabloid press play on those fears – gets lost in a different scuffle about the right way to address these voters. And here we are, three years on: a Conservative party and prime minister with a history of racism and Islamophobia have just achieved the biggest majority since 1987, and we still don’t know how to talk to, or about, those who voted for such a party and such a man.

This is the Clinton trap – the trap of “some, not all”. Any question about some of the preferences on display gets cynically collapsed into an attack on all voters – even when the “some” is explicit. Francois’ comic spluttering about a slur on 17.4 million people is a classic of the genre. There are many reasons one might vote for a party that has been endorsed by Tommy Robinson and Katie Hopkins. But it is not wild to assume that for some, xenophobia might be one of those reasons.

Play Video 0:34 John McDonnell: 'If anyone’s to blame it’s me, full stop' – video

Above all, it is a losing game to pretend that these qualities do not exist and aren’t being weaponised by the right, while trying to quietly and discreetly appeal to them. That way lies the dismal hedge of the Ed Miliband years, putting “Controls on immigration” on a mug and hoping it will slake voters’ appetite for some comforting nativism. There is a balance that can be struck between calling voters names and crudely pandering to them.

One defining feature of a populist political climate is a feigned preciousness about the motivations of voters. They are always pristine and unimpeachable, especially when the voters in question are “left behind” or authentically working class – a class that never seems to include black or brown people. En masse, these voters are seen to express a “will of the people”, which must not be questioned or challenged, only passively accepted.

That will – which takes shape in the years between elections, not on polling day – is in fact a composite of many factors. It is made up of ideological loyalty as well as a more abstract instinct that some politician is a wrong ’un. It is made up of information delivered by those parties who have greater access to media and more funds to do so. And yes, it is made up of human frailty, resentment and a yearning for status. The vision of Johnson’s Tory party is not really about Brexit – it is about renewing the status of “real” British voters via Brexit and its attendant cultural, social and economic meanness.

A Labour revival must tap into the energy for change on the ground | John Harris Read more

This strategy is so central for the right – and so powerful – that it must be addressed head-on, without falling into the Clinton trap. It is clearly not a solution to write off voters who respond to this sort of messaging, or call them names. But addressing them isn’t just about tone or language, it is about a vision to offer people who think nobody cares about them, a vision that we should have started building the moment that the first “Controls on immigration” mug cracked in the dishwasher.

It is an endeavour that needs to break the feedback loop between the rightwing media and the voter, get on the pitch and challenge nativist rhetoric, and move beyond feebly articulating anti- or pro-immigration positions. The reason people saw homelessness rise, the NHS on its knees and the deaths of thousands under austerity yet still voted Conservative is because they could not imagine an alternative. What is needed is that vivid alternative to the parochial nativism of a post-Brexit Britain, one that addresses those voters directly by making a pitch for their souls that goes beyond the economic. There were a few glimmers of that alternative during the last days of Labour’s muddled campaign, but by then it was much too late.

• Nesrine Malik is a Guardian columnist and the author of We Need New Stories: Challenging the Toxic Myths Behind Our Age of Discontent