Alistair Darling is a strange one. He was the chancellor who presided over the most catastrophic financial crash since the second world war and he was a senior member of the cabinet that subsequently took Labour to its second worst defeat since the first world war. Yet he emerged from the burnt-out wrecks of an economy and a party still commanding respect among his colleagues, his opponents and the media. This is partly because he is a decent, thoughtful, modest and essentially honest man – straight, at any rate, by the admittedly low standards that we expect of politicians. Those are qualities that shine out of this book. He is also regarded quite kindly because he is seen not so much as the villain of those calamities as their victim, a view he often takes himself in his account of the torrid thousand days that he spent at No 11.

It was his first misfortune to become chancellor just as the New Labour "economic miracle", which claimed to have replaced boom and bust with perpetual growth, was about to implode in the most spectacular crash since the 1930s. He writes compellingly about the market meltdown and ensuing recession, spicing the narrative with a droll wit and acidic observations about the "arrogant and stupid" bank chiefs. If this story has been told before, it is still informative to have the scary view from the edge of the precipice as Britain teeters on the brink of a complete collapse of its banks. On the critical night when the first rescue is being hammered out, he despairs of bankers who are still wrangling terms "as if they had weeks to seal the deal" when they were only hours away from shutting down the cash machines.

Why the crisis happened, and the risks of a repeat of it, are also the subject of Masters of Nothing. This stimulating book by two Tory MPs lays much of the blame on the irrational culture of the City for its recklessness and concludes with some thought-provoking recommendations for reform. For his part, Darling freely confesses that the tax revenues milked from the financial sector during the boom years left New Labour "blinded" to "ever greater risk-taking" in the City.

His second misfortune was to be chancellor when Gordon Brown was prime minister. On the face of it, they were a good pairing. Darling's personality is essentially placid, passive and self-deprecating. That could have been a good foil for the volatile, domineering and egotistical Brown. Darling was not a Blairite and he harboured no ambitions for the leadership. So Brown had no rational reason to have his paranoid tendencies aroused by his old friend. The two men, whose Scottish homes were just a few miles apart, had known each other for 20 years. Before they fell out, Maggie Darling would often babysit for the Browns. Yet her husband soon found that past loyalty and friendship were no protection from the dark side of the next-door neighbour. Brown replicated with his chancellor the horrendously dysfunctional relationship he had previously had with Tony Blair. A problem from the start was that Brown really wanted Ed Balls as chancellor. With characteristic charm, he said as much to Darling, who knew he was intended as "a stopgap appointment" having been told by Brown that he should not count on being at the Treasury for more than "just a year or so". Vicious spinning against his chancellor by "Gordon's attack dogs" became relentless after Darling told the truth that Britain was facing the worst economic prospects in 60 years. It culminated in Brown attempting to force out his chancellor: failing because Darling wouldn't budge and Brown was now too enfeebled to insist.

The Brown depicted here is a deeply unpleasant man. There are admirable qualities, and Darling acknowledges them, but they fail to redeem the ugly dimensions of his personality. Brown is the mafia boss of a "brutal regime". He is a cowardly capo, too, who gets his hatchet men to do his dirty work by briefing against members of the cabinet who have had the temerity to disagree with him. He is the hopelessly deluded prime minister in denial about the severity of the recession and incapable of grasping the economic and electoral imperative for Labour to present a credible plan for dealing with the deficit. No one in his inner cabal dares to tell him the truth: "Gordon was only told that which he wanted or could bear to hear." He is the self-pitying whiner who thinks the world is conspiring against him while "he seemed to have no conception of the effects of his sometimes appalling behaviour on those close to him, or the political damage his way of operating could cause".

The portrait of the Brown premiership as a combination of chaos and brutishness is already highly familiar from previous accounts. There is corroboration here rather than anything startlingly fresh. Coming from Darling, it nevertheless has sting, because the two men were once close. This is confirmation that Brown was incapable of forming stable and trusting relationships even with the most amenable of colleagues. Darling was famous among the rest of the cabinet for his stoicism. He writes: "My pain threshold is pretty high." Yet even this most even of men was "driven to breaking point" by Brown's abominable behaviour.

Some of us wrote when New Labour was still in office that the government was riven with intense and debilitating personal feuding and policy disputes, the most nasty of which revolved around Gordon Brown, and which became even more poisonous once he moved into No 10. The likes of Peter Mandelson and John Prescott denounced us as exaggerators, sensationalists or fabricators. I suppose it is too much to expect an apology now that the memoirists of the New Labour years have confirmed that the fear, the loathing and the psychological flaws were every bit as bad as we said.

Then there is the big question for the Labour party. Why did no one do anything? Why did Tony Blair, whose book makes clear that he viewed Brown as temperamentally unfit to be prime minister, not try to prevent his succession? Why, once it was beyond dispute that Brown was a terrible prime minister, did the cabinet not act? I tend to agree with Darling that Labour might have enjoyed a better fate at the last election "had we played our hand differently". So why did the cabinet not replace Brown with someone who might have saved Labour from defeat – or at least softened the scale of it?

Darling clearly would not have minded if someone else had wielded the dagger, but he would not do the deed himself, so he says, because of a "residual loyalty to him which I found impossible to overcome". Other members of that cabinet have offered a variety of different excuses for their paralysis. I could produce a complex analysis of the personal and political calculations that left Brown in place to lead Labour over the cliff. Perhaps, though, it really boils down to something much simpler: they were all too scared of him. Their party paid the price.

Andrew Rawnsley's The End of the Party is available in paperback (Penguin)