Very few people understand the Conservative Party. Among Labour activists, there is still a widespread preference for hurling abuse at it, and treating the word “Tory” as an obscenity requiring no explanation. The reasons the party has become the most consistently successful over the past two centuries, not just in Britain but in the world, and stands on the brink of a crushing election victory, are, on the whole, unexamined.

On the Conservative side, most people would much rather get on with electioneering than attempt to convey any deeper sense of their beliefs and assumptions. If pressed, they might fob you off by naming their Tory heroes, or by pointing out that the Conservatives have traditionally been pragmatic, adaptable and suspicious of abstract doctrines. They would certainly be hard put to give any coherent explanation in policy terms of the party’s success. The list of U-turns is too comprehensive, and recently grew longer when the leadership announced that it has appropriated Labour’s policy of an energy price cap.

As William Harcourt, the Liberal leader at the end of the 19th century, put it, “The Conservatives, mark my word, never yet took up a cause without betraying it in the end.” Membership of the European Union is the latest absolutely central policy that the party used to support and has now repudiated. Theresa May is hard at work dismantling the edifice constructed with such effort in the early 1970s by Edward Heath. Her demolition job is lent added piquancy by the fact she herself supported the Remain side in the referendum campaign. Yet now she presents herself as the bold, brave champion of the Tory Eurosceptics.

I saw her wowing them in her speech to the Conservative party conference in Birmingham last October, when she told them that Article 50 would be triggered by the end of March 2017, and assured them that after Brexit the UK will be “a fully independent, sovereign country”, no longer subject to the “jurisdiction of the European Court of Justice”, for “we are not leaving the European Union only to give up control of immigration again”.

This flourish gained her a standing ovation, during which the Eurosceptic who was standing beside me, and applauding heartily, turned and said: “I can’t remember the last time I heard a Tory prime minister give a speech in which I agreed with every word.” A few minutes later, I bumped into a prominent Leaver who declared himself happier than at any time since entering the Commons in 1992. Theresa May has reunited the party. The Remainers have nowhere else to go and cannot avoid being swept along on the new course.

May’s essential aim in all this is to keep the Conservative Party united, so that it and therefore she can remain in power. Naturally she does not say this, because it does not sound like a noble aim. Commentators collude in her silence by downplaying this motive and quite often ignoring it altogether. They consider such questions of party management to be a grubby and subsidiary matter. They also tend, with a few admirable exceptions, to know nothing about the party’s history.

But ever since the catastrophic split of 1846 over the repeal of the Corn Laws, the first duty of every Conservative leader has been to maintain unity. Here is the thread of continuity that renders the party’s behaviour comprehensible. Seen in this light, its U-turns and zigzags fall into place. We behold a tradition of practice and can stop searching in vain for ideological consistency.

Many Conservatives were and are swayed by various conflicting ideologies and the task of keeping them together is complicated. In 2005, the year of the party’s third consecutive defeat by Labour under Tony Blair, Geoffrey Wheatcroft published an enjoyable book called The Strange Death of Tory England, in which he claimed the party was “in terminal decline”, brought about by Margaret Thatcher’s “destructive contempt for traditional institutions” and “the ruthlessness of her class war”, shown by the removal of Etonians from positions of influence. According to Wheatcroft, as early as 1965, “the old governing class was losing its nerve, or its will to win”: in that year White’s club, in St James’s Street, was unable to find one of its own to run for the Tory leadership.

Yet within nine months of the book’s publication, David Cameron, an Etonian member of White’s, stood for the Tory leadership and won. The old governing class was displaying a greater adaptive capacity than Wheatcroft expected. Yet Cameron faced a dangerous problem, which none of his recent predecessors had managed to solve. His party was deeply split on Europe. In order to win the leadership he was forced to make a minor concession to the Eurosceptics (withdrawal from the European People’s Party, the grouping of centre-right parties in Brussels), after which he played for time and urged his followers to stop banging on about Europe. But by the middle of the last parliament, large numbers of Conservative voters and activists were deserting the party for Ukip (nearly four million people voted for Ukip in the 2015 general election), and this haemorrhage could only be staunched by making a major concession.

Cameron promised an in/out referendum on Europe. He did so in order to hold his party together. Both sides agreed to take their cases to the British people. An act of weakness was camouflaged as an exercise in direct democracy. And, in terms of party management, the manoeuvre has so far worked well. It kept the Conservatives united enough to win the 2015 general election. Cameron then lost the referendum: a result that surprised him, and led to his resignation and the eclipse of his entourage. However, Conservatives can thank him for leaving their party in a stronger position than any of its rivals.

The speed with which the Tories chose a new leader and united behind her surprised many observers: not that it should have done, for the same cold-hearted willingness to take tough decisions was observable in November 1990, when Thatcher, who had ceased to pay adequate attention to the task of keeping the party together, was replaced by John Major. Less than three weeks after Cameron had announced his resignation on the morning after the referendum, Theresa May was installed as Prime Minister.

She now seeks her own electoral mandate, and has chosen a moment that to all appearances could not be more favourable to the Conservatives. She maintains, of course, that a strong Conservative Party is in the national interest, because it will give her a stronger negotiating position in Brussels. She does not, however, add that she will be stronger because as long as she gets a considerably larger majority, she will not find herself at the mercy of a very small number of her own backbenchers, as Major did during the 1990s. One of the paradoxes of this election is that she is, in a sense, holding it in order to liberate herself from the hardline Tory Eurosceptics who may well decide that whatever Brexit deal she has done is unacceptable.

In leaping so nimbly from Remain to Leave, May can look back to, or unselfconsciously draw upon, a much longer Tory tradition of intelligent adaptation to circumstance. The party has never allowed itself to become trapped in some reactionary dead end, where it maintains the purity of its beliefs at the expense of becoming detached from events.

***

The word “Tory” is first found in English politics in about 1680, and originates in an insult thrown at the Tories by their enemies which came to be worn as a badge of pride. It derives from the Middle Irish word “tóraidhe”, meaning a pursuer, referring to the dispossessed Catholic Irish who harried and robbed the Protestant English settlers. In English politics, the Tories stood for church and king: the Church of England and the Stuart monarchy. They were royalists who looked back with admiration to the best of Charles I’s supporters. Their Whig opponents (whose name was likewise an insult, probably derived from “whiggamore”, meaning a horse thief and referring to Presbyterian rebels in the Scottish Lowlands) favoured parliamentary authority and greater toleration of Protestant dissenters.

The trouble with having political principles is that they can swiftly come into conflict with each other. Never was this more true than in 1688, by which time James II, after only three years on the throne, had managed to alienate the Tories who were originally among his most fervent supporters. Although he had promised to uphold the Church of England, he was a Roman Catholic and at once set about subverting the Anglican establishment. He was chased out of England and replaced by the Dutchman William of Orange, who had married James’s daughter Mary. William and Mary had the great merit of being Protestant, as did her sister, who would become Queen Anne. Parliament was so determined the country should remain Protestant that in 1714, after Mary and Anne had produced no heirs, it brought over the equally Protestant House of Hanover.

Roman Catholicism was associated with Continental absolutism: France’s Protestants, the Huguenots, suffered a ferocious persecution that prompted many of them to flee to London. Louis XIV of France showed that under a Catholic monarch, Protestants had no future. Unfortunately for the Tories, some of them still hankered for the restoration of the House of Stuart. They were dismissed by the Whigs as Jacobites: that is to say, traitors. They had committed the cardinal political blunder of remaining loyal to the losing side. Only from the 1760s did they start to revive.

To draw a direct line between this early Tory history and the present-day Conservative Party would require so many qualifications as to be of doubtful value. Tories have a reverence for Edmund Burke, and especially for his counterblast against the French Revolution, Reflections on the Revolution in France (1790), which expresses with incomparable genius the case for preserving and modifying the existing order rather than overthrowing it. But Burke, rather inconveniently, was a Whig.

One may identify, if one wishes, certain attitudes, including reverence for the nation, the established Church, private property and traditional sources of authority, which survive in the Conservative Party. To this day, you can tell a lot about someone’s politics by discovering whether he or she thinks that the French Revolution was a good thing.

Conservatives have long loved to trace implausible lineages back into the 18th and 17th centuries, usually in order to justify whatever they want to do now. But only after the passing by the Whigs of the Great Reform Bill 1832, which rendered the Commons more directly susceptible to the influence of public opinion and was followed by a general election in which the Tories suffered a drubbing (they were reduced to 175 out of 658 Commons seats), did political parties start to emerge in recognisably modern form, with the explicit aim of obtaining a parliamentary majority.

The Conservative Party was above all the creation of Sir Robert Peel (1788-1850), who had new money and lots of it. His father made a fortune as a cotton spinner in Lancashire, bought an estate near Tamworth in Staffordshire, became a Tory MP and told his son: “Bob, you dog, if you are not prime minister some day, I’ll disinherit you.”

To this end, the boy was sent to Harrow, where he excelled, and Christ Church, Oxford, where he took the first ever double First, the curriculum having just been divided into classics and mathematics, with undergraduates usually opting to be examined in one or the other.

He entered parliament as a Tory MP in 1810, was soon identified as outstandingly gifted, and in the 1820s became the greatest of all home secretaries. His many reforms included founding the Metropolitan Police and pushing through Catholic emancipation. Peel would reform anything when he thought the right moment for reform had come. The great mid-Victorian journalist Walter Bagehot wrote of him in a famous essay published in 1856:

No man has come so near our definition of a constitutional statesman – the powers of a first-rate man and the creed of a second-rate man. From a certain peculiarity of intellect and fortune, he was never in advance of his time. Of almost all the great measures with which his name is associated, he attained great eminence as an opponent before he attained even greater eminence as their advocate.

Peel opposed the Reform Bill but was far too intelligent to go on opposing it once it had been passed. The word “conservative” in its modern political sense is first found in an article published in 1830, and soon came into general use. The Carlton Club, which served in its early decades as the party’s organisational as well as social centre, was founded in 1832. Two years later, in 1834, Peel published the Tamworth Manifesto, the first document of its kind, addressed to the electors of his constituency but printed in the Times and other leading newspapers. Like most such works, it was intended to reassure rather than to amuse or excite. Peel confirmed that his party accepted the Reform Act as “a final and irrevocable settlement of a great constitutional question”. In the same year, he succeeded the Duke of Wellington as leader and pushed forward with what would now be called the party’s modernisation.

Wellington, who served as prime minister from 1828-30 and again very briefly in 1834, is sometimes described as the last Tory holder of that office, and Peel, prime minister briefly in 1834-35 and then from 1841-46, as the first Conservative. Some people like to attach different shades of meaning to “Tory” and “Conservative” while others treat them as synonymous, but for the purposes of this article it does not much matter which usage one prefers.



The point is that from the 1830s, modern political parties start to appear, and the Conservatives have been the longest-lived and most successful of these.

***

In 1841 the Conservatives secured, under Peel’s leadership, a decisive election victory, by 367 seats to 291. He became prime minister and assembled a gifted team of ministers including the young William Gladstone. Peel at this point still defended the Corn Laws, which kept the price of grain at an artificially high level. So, out of deep conviction, did the Conservative Party as whole, which represented the landed interest. But agricultural protectionism was coming under increasing attack from the Anti-Corn Law League, led by the radicals Richard Cobden and John Bright, who contended that it kept bread prices high, held back trade, impeded peace between nations, retarded agricultural efficiency and was defended only by “unprincipled, unfeeling, rapacious and plundering” landlords. Peel, who had a strong grasp of economics, was in private coming to agree with their arguments.

In the autumn of 1845, it became clear that the Irish potato crop had failed and a terrible famine was impending. In this desperate emergency, Peel decided that the Corn Laws must be repealed. With great difficulty he carried most of the cabinet with him, but his backbenchers were furious at what they saw as a contemptible betrayal. These country gentlemen distrusted Peel, the stiff and uneasy son of a manufacturer, but were not, for the most part, eloquent enough to take him on in the Commons. Benjamin Disraeli, the exotic author of some enjoyable but preposterous novels, no scion of the English gentry but the son of a Jewish man of letters, became their champion.

In his first great onslaught, delivered on 22 January 1846 after a long and dreary explanation by Peel had been received in dead silence, Disraeli compared the prime minister to a nurse who had dashed out the brains of the baby of protection, ie, the Corn Laws.

Peel was goaded into pointing out that Disraeli, despite now condemning his entire career, had once been willing to enter office with him. Disraeli responded with a flat lie, which Peel from some excess of scrupulousness declined to expose, despite possessing, and according to one account having with him in his despatch case, the letter in which Disraeli had solicited a job.

In June 1846, Peel got the repeal of the Corn Laws through, with the votes of the Whig opposition. Yet as leader of the Conservative Party he was finished. Disraeli had done his work so well that the party was nearly finished, too. It did not win a majority for another 28 years. Gladstone and the other ministers who had stuck by Peel were never reunited with Disraeli and the backwoodsmen who had destroyed him. Instead these Peelites were absorbed into the new Liberal Party, formed in 1859, which Gladstone would go on to lead.

Among Conservatives, the Peelite split has never been forgotten, and the blame for it has almost always been attached to the upright Peel rather than the slippery and impudent Disraeli. Lord Salisbury, who served after Disraeli for a total of 14 years as Conservative prime minister, said that to act like Peel was to inflict “the heaviest disaster the party could undergo”. Just such a disaster threatened in the early years of the 20th century, when Salisbury’s nephew Arthur

Balfour found himself threatened, as prime minister and Conservative leader, by a disastrous split between free traders and tariff reformers. “I will not be another Sir Robert Peel,” he declared, and tried to split the difference between the two, or “nailed his colours to the fence” as one of his friends put it.

In the short term, this tactic was a complete flop: in 1906, the Conservatives suffered a crushing defeat at the hands of the Liberals. Yet Balfour remained Conservative leader and was actually driven, in the general election of December 1910, to promise that if elected (they weren’t), the Conservatives would hold a referendum before introducing tariff reform, which was unpopular with working-class voters because it would raise the price of bread. So the desperate expedient of a referendum had occurred to a weak Conservative leader long before Cameron actually held one. In May 1945, Winston Churchill proposed a referendum to see whether the people would agree to the maintenance of his wartime coalition until after the defeat of Japan. Clement Attlee dismissed this suggestion as “alien to all our traditions”; Labour was understandably anxious to return to party politics.

Bonar Law, who succeeded Balfour and led the party through enormous difficulties from 1911 until after the First World War, understood, in the words of his biographer, Robert Blake, that “successive Conservative leaders had felt it their duty, at all costs and at almost any sacrifice, to avoid repeating Peel’s action”. The party must not be split. At the famous Carlton Club meeting of October 1922, when the Conservatives decided to withdraw from their coalition with the Lloyd George Liberals, Law told his followers: “I confess frankly that in the immediate crisis in front of us I do personally attach more importance to keeping our party united than to winning the next election.”

It is worth noting that in this period the Liberals allowed themselves to become split between the followers of H H Asquith and those of David Lloyd George, a setback from which they have never recovered. Indeed, the first person who split them, over Home Rule for Ireland in 1886, was Gladstone.

In more recent times, Rab Butler reckoned, when his ambition to lead the Conservatives was thwarted by Harold Macmillan (party leader and prime minister from 1957-63), that Peel’s splitting of the party had been “the supremely unforgettable lesson of history” for the Conservatives, and was an error that he, Butler, was not going to repeat. And in the 1990s, John Major and Douglas Hurd (who has since written a biography of Peel) strove to remind their ­fellow Conservatives of the disaster of 1846, though without achieving very much: the internal ructions continued and in 1997 Labour swept into power.

***

Even more important than these explicit references to Peel is the tradition of behaviour that has evolved within the Conservative Party thanks to 1846. It is considered entirely natural to get rid of a leader who no longer commands the confidence of the led. In 1940, this happened to Neville Chamberlain, who only two years before had been acclaimed as a great man.

The position of Conservative leader is an extremely precarious one. Of the 21 leaders from Peel to Cameron, so from 1834 to 2016, only two have managed to step down at a time of their own choosing: Lord Salisbury in 1902 and Stanley Baldwin in 1937. The others have all been forced out by the party, the voters or ill-health. Because the leader knows his or her tenure is precarious, he or she has the greatest possible incentive to be flexible: to adapt policy not just to changing events, but to shifting tides of opinion, both within and outside the party.

Many people imagine that to be a successful political leader, all you need do is work out the correct policies, promise to implement them and have the courage and honesty to keep your promises. Courage is indeed an indispensable quality in a leader. But the idea that an inflexible attachment to certain policies is the highest form of politics is intolerably naive. What works at one time becomes a perilous liability at another. There are moments, too, when an opportunity has to be seized before it can be exhaustively evaluated. Speed is of the essence. A sort of impudent resourcefulness is required.

Theresa May is acclaimed just now by her followers as a great woman, but no one can tell what her party will think of her in two years’ time. The ruthlessness of its defenestrations is unmatched by its rivals (consider the fate of Iain Duncan Smith in 2003), which is why it has outlasted them all. The party is an instrument for winning power by determining more quickly and surely than its competitors what the nation wants, and how to provide it. A party that was identified with landowners, industrialists and plutocrats became also from the 1880s the party of “Villa Toryism” in the suburbs, from 1951 the party that ensured the building of even greater numbers of council houses than the postwar Labour government had achieved, and from 1980 the party that sold off those council houses to their working-class tenants. A parliamentary party that as recently as 2005 was overwhelmingly white and male will after this election contain an even larger number of MPs who belong to ethnic minorities, and many more who are women.

Both Cameron and May have sought to accelerate that process by persuading good candidates from non-traditional backgrounds to stand. It would be convenient for Labour if the party had declined into a bunch of golf club bores, grumpily lamenting the way the world is going, but it would not be convenient for the Conservatives. Ruth Davidson, the Tory leader in Scotland, is confounding expectations mainly because she is a gifted advocate, who has mounted a strong defence of the Union. But she gets a hearing in part because she cannot be written off as a tweedy gentleman who has strolled in off a grouse moor.

I do not mean to pretend the story is one of unbroken triumph: the defeats of 1846, 1906, 1945 and 1997 were real enough, and so were many lesser setbacks. It is, however, unimaginable that the Conservatives would have stuck with someone such as Jeremy Corbyn, who has not only failed to win over the wider public, but does not even acknowledge, as one of the central obligations of his role, his duty to keep the party together. To Conservative eyes, Corbyn’s self-indulgence, and the willingness of his followers to indulge him, look grotesque. He seems bent on destroying his party without even having a great cause in view, one which is above merely partisan considerations, as Robert Peel did in 1846. l

Andrew Gimson is working on a book about British prime ministers from Walpole to May