Theresa May is right. “As Conservatives,” she told the BBC’s Andrew Marr on Sunday, “the arguments that we thought we’d had and won during the 1980s about the importance of free market economies – I think we thought there was a general consensus on that. And we now see that there wasn’t.”

I’ll say. As a consequence of her general election fiasco, the prime minister faces two formidable crises at the party’s conference in Manchester. The first is, to borrow a song title from Joy Division – the greatest band spawned by this city – she’s lost control.

No Conservative party conference is complete without a split or three. But the divisions yielded this year by Brexit and electoral embarrassment are of an especially vivid kind. When Nicky Morgan, a former cabinet minister and now chair of the Treasury select committee, feels she can say explicitly that Boris Johnson has no place in a “responsible government”, one cannot argue with a straight face that May leads a united party.

With his repeated interventions, the foreign secretary has put May in a position where her already dilapidated authority is an intolerable embarrassment to the Tories. He won’t shut up, and she won’t sack him. Nor, clearly, is she yet disposed to resign herself, or at least set a timetable for departure. The best she can hope for is that Johnson’s standing ovation on Tuesday does not amount to an acclamation.

What does unite the party is the conviction that there will be a leadership contest before the next election. And since the timing of that election is unknowable – the fragility of May’s deal with the Democratic Unionist party being what it is – it follows that this conference will be a squalid beauty contest. Amber Rudd, Ruth Davidson, David Davis and many others will be scrutinised like vegetables in a shop to see if they are fresh and appealing enough to make a difference.

Yet the identity of May’s successor is a secondary issue compared with the greater crisis that she and her party face – and which, unhappily for her, she understands more clearly than she is able to articulate to the public. It is a crisis of collective identity and purpose, a challenge to the very idea of conservatism.

Labour’s electoral rise has spooked the Tories because it challenges every preconception that most of them share about the world, its rule book and its ideological contours. It is a rich irony that the passage of May’s manifesto that most annoyed her party was also the most acute and prescient. “We do not believe in untrammelled free markets,” that much-maligned document declared. “We reject the cult of selfish individualism. We abhor social division, injustice, unfairness and inequality.” The election result and the national response to the Grenfell Tower tragedy suggest that the voters agree.

The problem is that most Conservative politicians do not. They loathed that assertion in the manifesto. They believe that the virtues of the free market are self-evident, unarguably so since the end of the cold war. They mutter that the facts of life are Tory, that what is needed is “real conservatism”, that turning up the volume will bring the public to its senses. To hear such talk is to watch a political gathering mimicking Jonestown.

I rather doubt that the nation has suddenly embraced socialism tout court in a moment of grateful epiphany. Last year, the British voted for Brexit after a campaign dominated by ugly anti-immigration rhetoric. This year, there was an electoral surge in favour of a proudly leftwing Labour leader.

What binds the two moments is an amorphous yearning for change. For today’s voters are both addicted to change and fearful of it. They love the fruits of globalisation, technological revolution, population mobility and automation – but they want a government that will address the pathologies of these thunderous forces. They crave novelty and security in equal measure. A contradiction? Such is human nature.

In this context, the Tories need to do much more than cobble together a routine retail offer. Tweaking tuition fees and increased funding for the help-to-buy loan scheme may generate a few headlines but, disaggregated, will make no impact whatsoever.

What the party needs now is not a shopping list of hastily assembled policies, but the shock therapy of absolute candour: it needs to be reminded of its first principles, of its potential to enhance the common good, and of the reasons why Tories are so often not trusted to act in the public interest.

When a consensus exists between two people or two million, it is possible to speak in shorthand. When that consensus breaks down – as it has – the dialogue needs to start again, in a spirit of humility. It is not enough, for instance, to say repeatedly that free market policies generate the prosperity necessary to fund decent public services.

That proposition needs to be explained in language that speaks to every home and those who long for their own home; to people who believe with every fibre in their being that they have no stake in the status quo, and that more Toryism means more poverty, more inequality, more misery.

Technocracy and the recitation of statistics will not do. This is a persuasive task that requires Conservatives to stop using the patois of the privileged. Ditch talk of “public service reform” and explain how Tory policies will improve the GP’s surgery in a voter’s neighbourhood, or the primary school at the end of her road. Dial down the obsession with “aspiration” that belittles those seeking only a decent life and more than 4p in their pockets.

Equality of opportunity is a fine objective – but declaring that you believe in it is not enough. Conservatives should be horrified by new research, conducted by British Future, suggesting that their failure to win over ethnic-minority voters cost the party 28 seats. In an interview with the House magazine, May tells Michael Howard that “modernisation” is “an argument that we had in the past … I think we’ve moved on from that.”

Again, she’s right. The party has indeed “moved on” from a strategy that broadened its appeal to one that is driving it steadily towards electoral exile. Do the Tories have the courage to confront the scale of the danger? On this most basic question hangs much more than the success of a single conference.

• Matthew d’Ancona is a Guardian columnist