Out on a limb: Cameron has cut himself off from core Conservative voters and is increasingly isolated. Photo: Dan Kitwood/Getty Images

It always used to be the case that the party conference before a general election was a time of uplift, real or imagined, for the Conservative Party. Even in the depths of misery – in 2000, say, or in 2004 – the party has shown an alarming capacity for self-delusion, deploying all that outdated tripe from the age of deference about the secret weapon of loyalty. There is little reason to suspect that this year’s events will be any different, given the level of delusion that appears to be afflicting the Conservative leader himself, and despite his embarrassments about Scotland, in failing to understand the realities of the issue, panicking when he did and almost certainly offering more than his backbenchers will allow him to deliver.

“Birmingham will partly be about celeb­rating what we have achieved with the economy,” a Tory MP tells me. “Our strongest suit is that Miliband and Balls don’t have a policy on it that makes sense.

“We inherited an economy in ruins and it’s no longer like that. Our unemployment is two-thirds of France’s, where they run a Miliband policy. But we have trouble on other fronts.”

Those fronts are, after the Scottish episode, what to do about England; what to do about Europe, especially its open-borders immigration policy; and what to do about the internal health of the party. “Some colleagues have lost a quarter of their members since the last election,” another MP tells me. “They mostly left over gay marriage and they aren’t coming back.

“Cameron said the issue would be forgotten by the election, but it bloody well isn’t. He had no mandate to do it and people won’t forgive him.”

The Conservative Party is in a state of deep uncertainty. Since the defection of Douglas Carswell to Ukip it has been accused of being on the verge of a split, but that, in fact, has already happened. Even if Tory MPs more conventional than Cameron have not formally turned against him, many have turned in their hearts. They have watched the haemorrhage, over the past four years, of activists from their constituency associations, people who have abandoned the party over disillusion with its policies and its failure to be more robustly and unapologetically conservative. They have become annoyed by what a former activist calls the “cringe” the party now routinely makes every time it is accused of being out of step with the centre-left consensus. Nowhere was this better seen than in the last-minute panic over Scottish independence, which culminated in Cameron’s much-derided “vow”, but that was not the only example. They have become frustrated by their leader’s fundamental disengagement from the real issues that affect their constituents – immigration, the lack of social mobility and the torturously slow economic recovery. When even tribal Tories become fed up with their leader – and many of them are even more disillusioned with him than their parents were with Ted Heath – the party ought to be getting worried.

Ministers still talk of how their party can win the election next May. There is the odd pessimist, but they mostly keep their own counsel. An older MP, no admirer of Cameron but tribally loyal to the party, regards many members of the 2010 intake as “flaky”. Outside the payroll vote, both in the Commons and in the Lords, gloom is more widespread. “We all underestimated Ukip,” a former minister and Cameron loyalist tells me. “It’s not just his fault.” Those fed up with Cameron – and there is quite a contingent now in the parliamentary party – point out that he didn’t help matters by piling insults on Ukip voters, many of whom often still voted Tory when it mattered. Some would like him to get on and conclude a pact with Ukip, but one of his friends asks: “How can you do deals with people you have called fruitcakes, loonies and closet racists?” Some people would begin by saying sorry, but Cameron doesn’t do sorry.

Part of the unhappiness is prompted by a doubt about how far he understands the mood of the party and the difficulty of its prospects, and how far he is thinking of the lecture circuit, or the merchant bank, or wherever he will consolidate his fortune when politics is over for him. His friends insist that, even if he manages to remain Prime Minister after next May, he’ll be out at midterm in the next parliament. Cynics regard this as his insurance policy against having to explain to Angela Merkel that Britain has voted to leave the European Union in a referendum that took place solely because his party forced him into it.

The criticism of his team in Downing Street, strident a year ago, has been muted a little by the effect of Lynton Crosby. The Australian pollster was welcomed by many backbenchers who knew about his grasp of populism. It is felt he has succeeded a little in getting Cameron and ministers to concentrate more on the issues that they believe voters really care about; but many MPs were shocked at the extent to which this unelected import influenced the July reshuffle, and particularly his role in Michael Gove’s defenestration. “Gove’s sacking was crass,” a backbencher tells me. “To be the enemy of the National Union of Teachers is a badge of honour. What we did to Gove was just appeasement.” However, a former minister says: “I don’t care what Crosby does so long as he wins us the next election.” MPs still feel that Cameron’s communications director, Craig Oliver, is second-rate and that the effective chief of staff, Ed Llewellyn, tells the Prime Minister too much of what he wants to hear. The party chairman, Grant Shapps, is felt to be more invisible than any chairman in living memory has been this close to an election. “He’s a complete lightweight,” a senior MP tells me. “He’s out of the loop and the big decisions are taken without him.”

Two factors are operating in Tory hearts and minds to prevent complete meltdown, but both are precarious. The first is the conviction that Ed Miliband is held in such contempt by the British people that, come what may, they will not vote for him to be their prime minister next year. His performance in the Scottish referendum campaign, in Labour’s heartland, reinforced that. Some Tories make a comparison with Neil Kinnock, and it is not intended to flatter. The second is the notion that all those former Tories who voted for Ukip in last year’s local government elections and this year’s European elections have “made their point”, and will be back en masse to do what is expected of them next May.

Miliband is certainly getting bad read-outs on his economic ideas, and if the pound in your pocket is to be the main factor at the election – which it could be – then he may struggle with that large, middle-class contingent that Tony Blair persuaded to come over to the Labour Party, and that is needed again. Whether, after recent events, he is a more comical, less credible figure than Cameron is a matter that will be keenly debated. Like her or loathe her, Mrs Thatcher was a politician on another planet compared to Neil Kinnock; Messrs Miliband and Cameron are altogether better matched, and neither is beyond being identified with disaster and failure.

The Ukip question is the thornier one. I know too many lifelong Tory voters (and even Tory activists) who say they will vote for the party come what may next year, to buy the line that they have all “made their point”. The point they wish to make – that the Conservative Party is insufficiently conservative – will for many of them not be made until Cameron has been punished for his apostasy, much as the Powellites punished Heath in February 1974, even to the point of bringing in a Labour government. One particular line of Tory delusion is that Ukip wants an in/out referendum on Europe, and yet it seeks to remove from office the only person who has promised to provide one and who stands a reasonable chance of being in a position to do so. As far as many Ukippers are concerned, the remedy for that is in Cameron’s hands: he can offer an electoral pact to them and everybody will be happy. But as lead delusionist, Cameron does not believe that such a pact is necessary, because he is sure the Ukip vote will shortly evaporate.

Much has been made, since he became leader nearly nine years ago, of David Cameron’s privileged background. Perhaps too little has been made of the extent to which he shares the characteristics of the last public-school-educated generation to run the party – the Edens, Homes and Macmillans. Then, the watchword (popularised by Macmillan) was “unflappability”: a good Tory leader, or even a bad one, never let it be thought he had made the slightest mistake, never apologised, never explained. A man who was never leader of the party – Rear Admiral Morgan Morgan-Giles MP – famously said, at a sticky moment for the party: “Pro bono publico, no bloody panico.”

Until the flap over the Scottish referendum, it seemed Cameron was cut from exactly that cloth; but then the panico in the last days of the campaign was precisely because of the suave, insouciant reluctance to be bothered by the prospect of the break-up of the United Kingdom until it seemed about to happen. If the Tory party were a band, it would have been playing “Nearer, my God, to thee” at that precise moment.

Douglas Carswell (left) unsettled the Tories by jumping ship on 29 August. Now he intends to becomes Ukip's first MP. Photo: Olli Scarff/Getty

The iceberg has been averted – just. Had it been hit, the party conference would have become a desperate showcase for a leader urgently having to prove to his party, and to those in the country who could be bothered to take notice, that he was still fit to hold his job. Instead, there will doubtless be an attempt to pretend that nothing is amiss, that it is business as usual, and that the party will sail ahead united and untroubled into an election campaign for which this conference season has been the first launch.

The problem for Cameron and his party is that it is already too late for that. The old toff unflappability cons no one any more, because the deference required to be taken in has long gone. The political sixth form at Westminster, former research assistants and apparatchiks to a man, is widely despised by the voters. It is all the worse for Cameron than for the others because of his abundant lack of humility, and the Bullingdon Boy picture that lurks in the back of too many people’s consciousnesses.

Ed Miliband may seem weird and Nick Clegg thick and duplicitous, but David Cameron has something far worse than that against him, namely his innate born-to-rule arrogance.

Now the Scottish diversion is over, the Tories must concentrate on fighting an election in just over seven months’ time. That a split in the party has already happened was evident not just in Douglas Carswell’s departing his safe Tory seat in Clacton for Ukip, possibly turning it into a safe Ukip seat in the process. It has been evident, too, in the scores of Tory MPs who have hinted they will be happy to state in their election addresses that they will campaign for a No to Europe vote in an in/out referendum, in the hope that they will not face a Ukip challenge. As things stand, they may just have to grin and bear it, because there is no sign of any sort of pact – yet. That may, of course, change after the Clacton by-election result on 9 October, especially if any other MPs are tempted to do a Carswell. However, had David Cameron read any history – and given that he thinks America fought with us in the Battle of Britain, he probably hasn’t – he would remember the outcome of the pact between Ramsay MacDonald, then secretary of the Labour Representation Committee, and Herbert Gladstone, the Liberal chief whip, in 1903 to allow some Labour candidates a clear run at the 1906 election, in return for Labour not fighting seats where the Liberals could beat the Tories. Within 20 years the Liberal Party was all but dead, and MacDonald was sitting in Downing Street.

The most serious allegation Carswell had to level against his former party leader when he defected in late August was that Cameron was “not serious” about change in Europe. He could have gone much further, without hyperbole, and said it was hard to detect any subject at all about which Cameron is particularly serious, except continuing to fulfil his student politician fantasy of being prime minister. The state of permanent chillax – as Harold Macmillan would not have called his distant successor’s pose of unflappability – also betrays a detachment from the serious issues that Cameron ought to be confronting on behalf of the country. Macmillan, too, had that detachment, which led to him having rings run round him in the Profumo affair, driving him out of office a few months later. Some Tory MPs – especially those who have worked in the private sector – comment on what short hours Cameron puts in. His recent bout of holidays, just as the Middle East went critical and Russia seemed about to invade eastern Europe, was Cameron to a tee, and his Scotland humiliation was but the first and most obvious consequence of it. If we didn’t know he loves power so much for its own sake, we’d think he’d given up.

Because of his arrogance, Cameron believes what he wants to believe and not what it is rational for him to believe. When he was leader of the opposition he thought he could repudiate the Treaty of Lisbon once he became prime minister, and repeatedly said so. Then, one day, enough senior diplomats told him that he couldn’t, because it was locked in to all the other European treaties Britain had signed since it concluded the Treaty of Brussels in 1972; that he had no choice but to believe them. In his bizarre reshuffle in July he dispensed with the services of Dominic Grieve, a serious and accomplished lawyer, as attorney general – apparently because Grieve persisted in telling him that he could not dispense with EU human rights laws without leaving the EU. Now that Cameron is Prime Minister he doesn’t have to change his mind, he just sacks the person who disagrees with him.

What used to be the Conservative-voting public – millions of them now appear to be the Ukip-voting public – is obsessed with Europe for one reason alone, and that is not (as is often represented) whether human rights laws allow Britain to deport people to countries where they might be tortured or face execution. It is because the EU allows the unlimited immigration of people from economically deprived countries in Europe. The voters believe these people take jobs that the indigenous working class would be happy to do (though the evidence for that is scarce), that they make an immediate claim on public services for which they have not made a contribution through their taxes (easier to prove) and that, in some cases, they behave in a way the indigenous population finds culturally offensive. “There has to be a firm policy on immigration before the election or we haven’t got a prayer,” says a senior Tory MP. “You should come on to the doorsteps and hear what people say about it. It can’t be fudged any longer.”

This is the key issue upon which Cameron has failed to make common cause with his party, which now widely finds his occasional bursts of rhetoric on the subject downright dishonest. But there are others. For reasons rooted in cynicism, Cameron pushed through same-sex marriage. The grass roots of his party overwhelmingly opposed it. For doing so, they were accused of homophobia. Doubtless in some cases that was true. In many others it wasn’t. Anyone who knows the Conservative Party knows that for decades it has included a proportion of homosexual members that probably exceeds the proportion of homosexuals in the population. They are not just in parliament, but in the salaried organisation and the constituency parties, and they have been for as long as most can remember. For most Tories their sexuality is an irrelevance. The opposition was grounded in four things: first, many thought the term simply shouldn’t be applied to same-sex couples for logical reasons, as it had only ever been applied to heterosexual couples; second, it was viewed as cynical pandering to the agenda of the militant left; third, there were many worse things wrong with the country that required not just the time, but the goodwill of Tory MPs, to get through; and fourth, Cameron had no mandate to do it. That goodwill is in short supply in the Commons now, and even shorter in the emaciated constituency associations. As usual, Cameron didn’t consider the consequences.

Yet, for all that, it is far from a delusion to suppose the Tories could, if not win the election next year, end up as the largest party in a hung parliament. Some of the uncertainties that afflict the Tories also afflict Labour. In areas heavily populated by the white working class, notably along the Thames Estuary and in other parts of the south-east, Ukip is making more inroads into the Labour vote than into the Tories’. Ed Miliband is not popular with the general public, nor is he viewed as especially effective. He has no entrée with the middle classes of the sort Tony Blair had. He has no compelling narrative about the economy – the Tories can at least point to a record of growth, lower unemployment and some attempt to get the deficit under control compared to the mess they inherited in 2010.

“I’ve got white working-class constituents who’ve always voted Labour, but they won’t be voting for Miliband but for Ukip, because they feel he couldn’t care less about them,” a backbencher said. “So the results could be pretty random.”

“If we engage in fratricide at Birmingham we’re writing our own death warrant,” another MP told me. “But when it’s over we’re going to have to deal with a constitutional crisis. There’s going to be a lot of trouble over Scotland, and it will go back to Cameron’s poor judgement in allowing such a long campaign and then panicking at the end of it.” With numerous MPs now intimating they will never vote for the fulfilment of Cameron’s “vow”, more will follow, and the battle over that must not be underestimated. If there is a sense of calm at Birmingham it will be entirely artificial, and it will come before the storm.

Simon Heffer writes for the Daily Mail. His latest book is “High Minds: the Victorians and the Birth of Modern Britain” (Random House, £30)