It is ten years since Robin Cook died and somehow the hole left by his departure only seems to have grown bigger with time. Looking back on a decade of almost continuous crisis and decline for Labour, it is hard to think of a moment when the party would not have benefited from his presence. Ed Miliband would certainly have benefitted from his wisdom and support during five years in which Labour’s surviving grandees offered very little of either. Perhaps his most important contribution would have been in Scotland where he would have relished the opportunity to make the radical case against separatism and provided a living rebuttal of the SNP’s “Red Tory” attack line that did Labour so much damage at the general election. He was the missing link needed to reconcile Labour with its disenchanted former supporters.

Cook is still missed because he represented something that now seems to be lacking on the British left. The substance of this is the topic of a new book, Robin Cook: Principles and Power, written by his former media handler, John Williams. It tells the story of Cook’s time as Foreign Secretary and his struggle, as Williams summarises it, “to reconcile principles with the democratic necessity to compromise”. It records, in particular, his determination to integrate an ethical dimension into British foreign policy at a time when the realities of international diplomacy and the constraints of collective cabinet responsibility presented significant obstacles.

Williams provides valuable insight into the inner workings of government and the relationship between its key personnel. But the book’s real value is that it relates an important and previously untold part of the New Labour story that helps to shed light on how the party got to where it is now. Cook regarded himself as a moderniser by instinct, yet he was always clear that the goal of modernisation should be to make progressive politics work. While crediting New Labour with genuine achievements, especially in its first term, he felt that the party had sacrificed too much of its purpose in the quest for office. His willingness to press the issue became a recurring point of tension in relations with Tony Blair and other senior colleagues.

In the end the Blair government became too narrow and too contemptuous of progressive concerns to accommodate Cook’s brand of politics and he ended up resigning on an issue of principle over Iraq. While his standing was enhanced by the split, New Labour was irreparably damaged by it as its reputation for cynicism became embedded and the broad electoral coalition that delivered two landslides started to crumble. The aftershocks are still being felt now in the collapse of Labour in Scotland and the failure of Blairite candidates in two successive leadership contests. There will be more to come if and when the Chilcot enquiry finally reports.

Despite the plaudits he received at the time, it is important to record that Cook regarded his resignation as a personal failure. Although he rejected the idea that power should ever be an end in itself, he had little time for the argument that good intentions are all that count. The true conviction politician should never embrace opposition as a way of life because, just like the unprincipled politics of triangulation, it means giving up on the possibility of change. Radicalism requires a strategy for power. This matters today because Labour has emerged from an era in which the balance was often tipped too far in one direction and is now in danger of tipping it too far in the other.

The undoubted advantage of Jeremy Corbyn’s leadership is that it draws a firm line under the New Labour years and forces the party to think afresh about what it really stands for. Ed Miliband tried valiantly to achieve a similar reset, but his style was so diffident that only those threatened by it really noticed. Now that Labour is compelled to address some fundamental questions, it has a chance to move on, provided it comes up sensible answers. The problem is that Corbyn and his supporters have yet to give a plausible account of how they intend to build an electoral majority behind policies that have always been rejected by voters in the past. They too need to understand the democratic necessity to compromise, both with their internal Labour critics and with the country as a whole.

The most difficult challenge for those opposed to Corbyn’s approach is to be honest in accepting their own share of responsibility for his rise. It is easy to rail against the “politics of illusion”, but that is what you get when the politics of disillusion has run its course. There is no future in a bloodless technocratic centrism that ignores the country’s biggest social and economic problems in the name of “realism”. That much we know. The big question is whether Labour moderates can learn from their failure.

The evidence of Peter Mandelson’s recent leaked memo, in which he bemoaned “the last five years’ of intellectual sterility”, is not encouraging. A frank assessment of where Labour went wrong would admit that its period of intellectual sterility began twenty years ago, under him. Telling Corbyn that he is wrong because he can’t win simply confirms the impression that nothing has really changed. Labour’s recovery can’t start with a brute assertion of electoral logic. It has to start with a principled debate about the best way to achieve its goal of a stronger and fairer society. If it goes back to believing that winning elections is all that matters, it will fail even on this narrow measure of success. A hollowed out Labour Party with a minimalist offer cannot secure the broad support needed to govern.

Corbyn’s campaign only achieved lift-off because the Labour leadership contest seemed to be in danger of becoming a mindless stampede to the right. The mainstream candidates spent its opening days trying to outbid each other in the number of progressive commitments they proposed to dump – the mansion tax, the 50p higher rate, responsible capitalism – as if Labour could hope to win in 2020 by standing for less rather than more. They did this reflexively, in the erroneous belief that Labour lost because it was too radical. It was only after Corbyn had become the shock frontrunner that any of them sounded like they wanted to lead a party of the left. It’s hardly surprising that Labour’s membership didn’t buy it.

Many now expect Labour to descend into civil war, as early skirmishes over defence and fiscal policy would seem to confirm. But there is another scenario in which its different wings attempt to reach an historic compromise that allows them to marry principles with power in the way that Robin Cook envisaged. Corbyn and his supporters would learn to embrace compromise as a principled democratic choice while party moderates would move beyond narrow electoralism and attempt to recapture a genuine sense of idealistic purpose. Cook would have been a good rallying point for a project of that kind. Labour’s prospects over the next five years – perhaps its very survival – now depend its ability to find a new progressive consensus without him.