Once again a contest takes place to elect a leader of the opposition, perhaps the toughest job in British politics. You might think the prime minister faces more varied responsibilities and nerve-shredding decisions, but the person across the despatch box has a task that is much less clearly defined. He or she cannot be judged by the implementation of policy, only by words and deeds. The role is closer to an art form. The leader of the opposition must appear “prime ministerial” in order to succeed and yet has none of the levers available to the actual prime minister.

Oddly there is little written about this strange job or how to succeed in it, which is quite an omission when so much turns on the right pick. Many fail in the task, a few succeed, and whatever the personal differences may be between these two groups, you can bet they’re not captured by the sort of labels—“continuity Corbyn,” “clean break,” “Leave-curious” and “hardline Remain”—which are shaping the current contest in the Labour Party.

Judging whether candidates have a capacity to lead matters because the UK has, for some time, been close to a presidential culture in which the media focus is almost entirely on the prime minister, and their rival on the opposite bench. Yet shelves creak with volumes on lives of prime ministers. The same shelves are unburdened by weighty tomes narrating experiences from those on the other side of the battle.

This is partly because a lot of opposition leaders fail to become prime minister and are disinclined to reflect at length on what went so traumatically wrong. As Neil Kinnock once put it when asked if he planned to write his autobiography: “Memoirs are for winners, not losers.” Kinnock lost two elections, in 1987 and 1992 without ever becoming prime minister, just as Michael Foot had lost before him, and Ed Miliband and Jeremy Corbyn have done so since. On the Conservative side William Hague, Iain Duncan Smith and Michael Howard likewise toiled in vain. Of this large group only Foot wrote about his experience in a brilliant book, Another Heart and Other Pulses. Characteristically, the book was more about his personal heroes such as Byron than it was a study of his own torrid and doomed phase as leader. For the rest, after the invariably noisy spell in the job itself, it is more typical to leave only silence.

This is a huge gap. Hazy as the job may be, it is of enduring significance. Going back to Gladstone, Disraeli and beyond there has always been an identifiable leader of anti-government forces in the Commons, and the job has enjoyed official recognition for over 80 years. Since 1937 the leader of the -opposition has received a state salary in addition to his or her income as an MP. The holder also receives a chauffeur-driven car.

The extras are part of the strange contradictions that both glamorise and torment a leader of the opposition. There are the grand perks, like car and driver, and there are considerable resources for a leader’s office, financed by the taxpayer. Then there is the privilege to ask the prime minister six questions most Wednesdays of the year and command the stage at other big moments, including party conferences and budget day.

Yet this spot in the limelight can be a form of hell. Prime Minister’s Questions is stressful. An opposition leader who chooses the wrong topic or fails to “win” is in trouble. The one clip picked for News at Ten used to be seen as make or break, but now instant verdicts from the commentariat on Twitter can have the backbenchers restive before the session is even over. Responding to a budget is a special nightmare. There is no advance notice, and inevitably baffling surprises occur that require quick thinking. If the leader is clunky rather than agile, he or she loses self-confidence and the confidence of colleagues. As for the party conference stage, those who have mastered it are still frantically nervous; and those who aren’t masters have powerful reason to be. It was days after an unfortunate conference address (“the quiet man is here to stay, and he’s turning up the volume”) that Duncan Smith faced a fatal parliamentary coup. Miliband never quite recovered from a speech in which he forgot the “deficit,” the theme his opponents and the media had decided was paramount.

Opposition leaders have some limited influence to shape events. The best recent example is probably the stand Miliband decided to take against air strikes on the Assad regime in 2013, which—with the help of Conservative rebels—saw the government defeated, the British policy withdrawn and pressure build for a similar vote in Washington which eventually saw Barack Obama back off too. In the later days of New Labour, Cameron—then in “liberal” mood—successfully joined forces with government rebels to stand against certain draconian steps, such as extended pre-trial detention of terror suspects.

But such instances are rare exceptions. Especially with the sort of majority Boris Johnson now enjoys, the government can almost always whip its way through. Most of the time, the government can legislate as it likes, and all the opposition can do is oppose impotently. As Tony Blair once put it: they can speak but they cannot do. Yet an opposition leader must always give the impression of momentum and purpose.

No wonder many have failed in the task. But there are lessons to be learned from them, and those who have succeeded. In the last 60 years, the phase in which first television and then social media became powerful forces, there have only been five outright winners straight from opposition—fewer than one a decade. These were Harold Wilson (1964), Edward Heath (1970), Margaret Thatcher (1979), Tony Blair (1997) and David Cameron (2010).

Put Heath to one side as a very rare exception—a successful re-run in opposition politics. Having been slaughtered in the 1966 election, he was lucky to get another chance against a government which, by 1970, had lost all drive and sense of purpose.

The other four possessed the essential qualification to be a successful leader in opposition. They were political teachers, constantly making accessible what they were trying to do. Above all, they answered the all-important “Why?” question, always explaining why they acted in the way they did. They framed an argument, and then made apparent sense of their objectives with a few carefully chosen policies.

In the early 1960s, Wilson spoke of the “Britain that is going to be forged in the white heat” of the technological revolution, and of the need to “harness socialism to science.” The verb was as important as the appeal to the future. Many politicians pose as “modernisers,” a vacuously apolitical term, but “harness” implies that active government could speed the change.

Thatcher was an instinctive teacher, her greatest quality—even if some of her teaching was nonsense. In the late 1970s, she made the complexities of monetarism accessible by citing her father, who never spent more than he earned while running his shop in Grantham. The state is incomparably different to a shop, which cannot print its own currency for starters. But here at least was an account that people could follow of how Britain had come to be, in a phrase of the time, “the sick man of Europe.” Why did she want to cut some public spending? Look what happened to the 1970s Labour government not following her father’s prudence. Cameron and George Osborne used similar arguments to justify their turbo-charged Thatcherism in the immediate aftermath of the 2008 crash, the only leaders of a mainstream party in the western world to argue for real-term cuts in spending. Why? Well, the reckless Labour government had “maxed out the credit card.” They had offered an explanation, albeit a dubious one, in vivid language, and virtually every media outlet in the UK soon followed them in making eliminating the deficit the defining test of leadership.

“Blair was an around-the-clock teacher”

Blair was an around-the-clock teacher after being elected Labour leader in 1994, largely about the way his party had changed—he never tired of explaining that he was a man of a new “radical centre.” If voters wanted change, he was radical. If voters sought reassurance, he was on the “centre ground,” and could be trusted not to do anything risky.

All the explanations raised myriad questions—but the election-winning leaders tried to make sense of what they were doing.

There are also lessons to be learnt from leaders who couldn’t teach. Labour’s new leader will be taking the reins from a man who didn’t even try. Corbyn had spent his career campaigning from the backbenches, addressing meetings where the audiences agreed with him. He had not been required to explain until, relatively late in life, he took centre-stage, and found himself unable to do so. In the 2019 election, he announced a bewildering hailstorm of new policies, for which no argument had been prepared in advance, as if free broadband and a milkshake tax were, in themselves, a form of explanation. When the programme did not boost the party’s support the leadership announced still more policies, as on women’s pensions, complacently assuming that the voter would somehow grasp why they were proposed. There was a powerful argument buried away about the “good that government can do” but it was not made.

Kinnock had the capacity to teach and the instinct to do so. At his peak he was a mesmerising speaker, but as a leader he taught less and confronted more. Arguably—with a divided party, a serious challenger in the form of the SDP and the rightward drift in ideas—he had no choice, but it eventually came at a cost.

At first when Kinnock took on his party, he appeared to acquire greater authority. Commentators praised his “courage.” But soon he noted that much of the media wanted a “shoot-out at the OK Corral” every week of the year. After he lost for the second time in 1992, he reflected that voters saw a leader at war with his party, not a prime ministerial figure. His most famous speech at the 1985 party conference, in which he attacked Militant-led Liverpool for hiring taxis to deliver redundancy notices to its own workers, illustrates why. In the same speech he made a considered explanation as to why it was possible for government to intervene in new and different ways, and raised the possibility of an “enabling state” which helped people realise their potential, countering Thatcher’s portrayal of the state as stifling constraint on “freedom.” But no one noticed because when conflict rages it will command the headlines. While Kinnock the warrior was established, Kinnock the teacher was lost.

Like Kinnock, Hague could have been a political teacher. He was an orator and possessed that currently underused political weapon—wit. But he never decided what it was he sought to explain. He made a fatal error at the offset, in 1997. The Labour government had pledged to stick to the Conservative spending plans and not raise income tax. Blair and Gordon Brown adhered to their pledge, although the outgoing chancellor, Ken Clarke, later revealed he had no intention of implementing such “eye-watering” constraints. Yet Hague foolishly described Labour’s approach as “profligate and reckless.” He would have caused far more trouble for his opponents had he declared his support for them for carrying out Conservative policies. Later Cameron was cleverer, sometimes undermining Blair by backing him.

Here is another lesson. Opposing a government is not always in the interests of the opposition. Some Labour MPs would have twitched nervously if they’d seen Hague and his colleagues backing a Labour government’s economic policies after 18 years of Tory rule. In his pre-austerity mould, Cameron stirred up discomfort on the Labour side when he enthused about the decently-funded, mixed economy Blairite approach to public services. The provocative suggestion was that in schools especially, he merely wanted to see through reforms that the prime minister considered to be necessary, but which his party would dilute. Cameron undermined Blair by supporting him.

There is one other lesson from the recent past. If a leader is elected without the support of most of his or her MPs, storms will brew. Duncan Smith never topped the ballots of MPs in the party’s 2001 contest and yet won in the final vote of members. The party’s MPs removed him two years later. Ed Miliband was subject to hostile briefings and embryonic internal coups from some MPs who could not accept he had won—his brother had topped the poll of MPs. Corbyn could never count on the support of the parliamentary party—four-fifths of whom voted against him in a vote of no confidence just 12 months into his reign. Voters do not follow politics closely but they do notice something is up with a leader if their own MPs are not supportive. A leader of the opposition greatly at odds with MPs will not win, a lesson for stroppy MPs as much as leaders.

Soon there will be a new Labour leader of the opposition. On one level the impossible job looks even more impossible. Unless the victor is Keir Starmer, who attracted 88 parliamentary backers, none of the other candidates enters the final strait with more than 17 per cent of MPs behind them. More fundamentally, Labour has lost parts of the north and Midlands, as well as being obliterated in Scotland. The party has lost eight of the 11 general elections since 1979.

“The task is less daunting than some Labour leaders have faced”

But the task is less daunting than some Labour leaders have faced. Much of the fuming tension under Corbyn was over the past rather than the present, the conduct of the man when he was a sometimes-careless campaigner with no ambition to be on the frontbench, let alone leader. Conversely, some of Corbyn’s supporters were disdainful of anyone associated with the New Labour era. Yet on most contemporary issues there had been a greater coming together than either side dared to notice. Corbyn compromised on his previous support for unilateralism and Euroscepticism, while his internal opponents accepted or actively supported some of John McDonnell’s ideas. Now there is also something of a consensus over what went wrong: the inept handling of anti-semitism, the indiscriminate eruption of policies at the election, strategic incompetence. As an important bonus the next leader will make his or her moves as a Tory government debates how much more the state can do. The tide moves leftward even as the party of the left has lost spectacularly. An army of commentators will urge vaguely but relentlessly the next leader to move to the “centre ground.” It has not noticed the direction of the tide. A leader must show he or she is part of the sea change and, although in reality virtually powerless to change it, must at least appear to be directing the rhythms.

Evidently, the next leader of the opposition needs to change a party that has been unable to win. But that does not mean a new leader must theatrically be seen to be taking on his or her party relentlessly, even though that will be the demand of Times editorials and a test of some BBC editors. Instead he or she must prioritise being in a constant dialogue with the wider electorate, explaining endlessly and accessibly what they stand for and why. Having secured an audience policies must follow, proposals that reinforce the explanation. Political responses to the government and unexpected events must be deft.

Currently the prime minister and his team act with a justifiably confident swagger. The confidence will soar to higher peaks if the next leader of the opposition proves to be useless. But the swagger will turn to panic if the opposition leader leaps ahead in the polls. In order to have any chance of making such a leap the new leader must learn the subtle and sometimes counterintuitive lessons from the recent past.