A man of his word: Gerald Scarfe’s Sabre-Toothed Ptorydactyl, SDP dodo and Lesser Welsh Speckled Dragon (Kinnockus Pillockus Rhetoric), 1989

When Neil Kinnock told Ed Miliband that he should consider standing for the Labour leadership, the latter mentioned the little matter of his brother, David. Kinnock told him: “If you don’t mind David standing against you, why should David mind you standing against him?” “It’s not that simple,” said Ed. Kinnock replied: “Nothing is ever that simple.”

Kinnock laughs as he tells the story, but it’s a solemn laugh. In the 31 years since he was elected Labour leader himself, nothing has been simple for Neil Kinnock. If you listened to Tony Benn or Arthur Scargill, he was the great betrayer. Others believe that he has saved Labour twice. He saved it when he was leader from becoming an irrelevance after the Benn ascendancy. And then by helping to make Ed Miliband leader he saved it from becoming a mere US-style electoral machine whose default position was to support the rich and powerful. Today, as Ed Miliband goes into a make-or-break conference in Manchester, Kinnock is still prominent in his corner.

Is Ed Miliband the Kinnock de nos jours, as is often said by his disparagers? Not at all, says Roy Hattersley, who was Kinnock’s deputy from 1983 to 1992. “They are completely different. Neil is the orator, the Nye Bevan figure. Ed is the cerebral politician, the Hugh Gaitskell figure.”

Back in the 1950s, Bevan personified Labour’s left and Gaitskell its right. Hattersley treasures the moment when Ed Miliband told him: “I’m a Croslandite,” because this establishes him in the Gaitskell camp.

But Kinnock says: “It’s a different generation with new challenges. There are lots of issues he [Miliband] doesn’t have to address. The conditions facing me required a bare-chested approach. It now needs greater subtlety.”

It would be hard to imagine two more dissimilar personalities. Kinnock is noisy, extrovert, affectionate and emotional, and he was, as Hattersley says, “the best platform speaker of my generation”. No one accused him of being geeky. By contrast, Miliband is quiet and thoughtful, charming but no tub-thumper, probably less sensitive to criticism than he sounds, and very clever. He told Kinnock that he was going to address Labour’s conference without notes. “I was terrified,” says Kinnock, but: “He does it brilliantly. It’s a sign of real depth.”

However, their leaderships have this much in common – the media and some of their party colleagues went in hard and early to kick the bounce out of them. The Bennites (for Kinnock) and the Blairites (for Miliband) predicted that neither would be prime minister. With Kinnock, it worked. It won’t work a second time if there’s anything Kinnock can do to prevent it.

I first met Neil Kinnock in 1982, when I was Labour’s press officer for the Peckham by-election at which Harriet Harman entered parliament. The grim, sectarian chairperson of the Peckham Labour Party women’s committee watched Kinnock exchanging rugby reminiscences with a senior policeman sent to look after our walkabout, and muttered furiously: “How macho!”

He had already made enemies. His refusal to support Tony Benn against Denis Healey for the deputy leadership of the Labour Party in 1981 had cost Benn the job. Benn never forgave him. Even in old age, in the final volume of his memoirs, published last year, Benn could not bring himself to mention Kinnock without a cruel sneer. He records attending an event and seeing guests who included “Neil Kinnock, now 65, laughing boy Kinnock”. Kinnock’s ready laugh is one of his most attractive traits.

In March 1983 the Labour Party lost one of its safest seats, Bermondsey, in south-east London, in a by-election. (It was won by Simon Hughes for the Liberals against Peter Tatchell.) The day the dreadful result was declared, Labour strategists, in a spirit of bravado, called another by-election straight away instead of delaying in the hope of better times. I again found myself in demand as a press officer.

One cold day that month, I went to Darlington station to meet a short, sturdy, noisy young man with ginger hair and a loud houndstooth suit. He stepped smartly off the London train, borrowed £5 from me and took everyone he found in the nearby party headquarters to the pub. In Darlington in 1983 you could buy a substantial round of drinks with £5.

He did not stop talking for a moment in the pub. As a stream of ideas tumbled out of him, each one perfectly wrapped in an evocative phrase, the ageing Labour grandee Barbara Castle listened in uncharacteristic silence. She seemed to be seeing the reincar­nation of her dead hero, Aneurin Bevan.

Eventually we dragged Kinnock across to the biggest hall in town, which seated hundreds and was packed out. He spoke for a full hour, making his audience laugh and cry in turn at his attacks on Margaret Thatcher’s government. “Now, the cabinet wets – By the way, do you know why they call them that? It’s because that’s what they do when she shouts at them.”

Then I took him to a private room and whispered that I had overheard a plot to oust Michael Foot as leader. Kinnock buoyed me with his optimism, and for a good five minutes we both believed that soon Foot would be prime minister.

We went down to the hotel bar, where a huge press pack was assembled; this was certainly the last by-election before the general election. There he took them on. This journalist had written a load of rubbish about Footie; that one was so deeply in hock to the Tories that he played “See the conquering heroine comes!” every time Maggie wiped her feet on him. As others left for their beds one by one, Kinnock’s musical Welsh cadences followed them to the hotel lift.

On the train home the next day, he snagged the houndstooth suit on a nail. No one has seen Kinnock in a loud suit for years. Seven months later, after Labour’s crushing defeat in the general election, Kinnock was elected leader of his party. He was 41 and his best years were over.

At once, the Conservatives, the newspapers and the Bennites set to work on him. Those who remember only the party leader, struggling to find words sufficiently boring that they offered no hostages to fortune; or the demoralised figure who gave up the leadership after his second general election defeat in 1992; or the European commissioner mastering the dead language of Eurocracy, stare in disbelief when I tell them that in the early 1980s he was easily the most exciting politician in Britain.

Born in 1942, Kinnock was the political leader of the Look Back in Anger generation. John Osborne’s play premiered in 1956, the year Britain invaded Suez and learned she was no longer a world power, and the Soviet Union invaded Hungary and showed that communism was no longer a proper vehicle for the idealism of the young. It was the generation to which both Osborne and Arnold Wesker spoke, a generation that believed it owed a debt to Clement Attlee and Aneurin Bevan; that spoke of socialism and Jimmy Porter’s great, brave causes. Youth found the voice that had been stifled by the grey conformism of the postwar years; a voice that had not yet turned into the muddled strivings of the late 1960s and the harsh sectarian stridency of the 1970s.

Yet he also carried Labour’s history as no other politician of his time could do, a man of the left with the revivalist socialism of the Welsh valleys. Michael Foot saw him like that, for Foot was Aneurin Bevan’s friend and biographer and Kinnock’s friend and mentor.

At the 1970 general election, aged just 28, Kinnock became MP for Bedwellty, one of the safest Labour seats in the country. After Labour’s 1979 election defeat, he entered the shadow cabinet as education spokesman, still aged only 37. If we had been told then that, 35 years later, Neil Kinnock would never have occupied any post in government, and that selective education would be stronger than ever, we would have laughed at so absurd a notion.

“Education is something he feels very deeply about – he knows it changed his own life,” Fred Jarvis, the then general secretary of the National Union of Teachers, told me. “And he’s not afraid to say the word ‘comprehensive’, which makes him unusual in today’s Labour Party.” Jarvis is still one of Kinnock’s friends.

He was, for those days, young and inexperienced when he became Labour leader in 1983, and felt in need of advice. A posse of clever men and women a decade younger than himself, politicians of the harsher era of the 1970s, became his praetorian guard: the Cambridge graduates Patricia Hewitt and Charles Clarke, who had been president of the National Union of Students (NUS); the Oxford graduate Peter Mandelson, who had been chairman of the British Youth Council. Kinnock says they “had been student union officers very young, and after that they came and worked for me”.

Among the other Kinnock confidants who had cut their teeth in student politics were the former NUS president Jack Straw and John Reid, a former young communist. Fighting the ultra-left was in their bones. They had fought them in the student unions and now they fought them for Kinnock, unrelentingly and obsessively.

They told him he had to acquire gravitas. Hewitt erected a barrier between their leader and the lobby journalists, who were used to the old, genial, witty Kinnock. Roy Hattersley says that Clarke did Kinnock great harm by protecting him when he did not need protecting.

“It was a terrible mistake,” Hattersley told me. “Clarke told him not to make speeches, because he only needed to make one mistake to lose the next election. But he would not have done that. He knew how not to do that. So Kinnock, this great platform speaker, just stopped using that talent. It was a dreadful waste.” (Clarke declined to be interviewed for this article.)

Kinnock’s protectors told him that his personality was all wrong. The noise, the passion, the bons mots, the houndstooth suits – everything that had endeared him to the public before he was elected leader – it all had to go. He started to wrap himself up in grey flannel suits and grey woollen phrases. Brendan Bruce, the Conservative director of communications for part of Kinnock’s time as leader, has said: “He was badly let down by his image-makers in recent years. There were endless things they could have done.”

It seemed as though they did not trust their man not to bungle if let out without a leash, and perhaps Kinnock started to take himself at their estimate.

Vicious attacks from right-wing tabloids and left-wing rivals punctured his skin, which turned out to be thin for a top politician. The magic went out of his relationship with journalists. His sure broadcasting touch started to desert him as he began to weigh every word and waffle to cover policy divisions. Perhaps for the first time in his adult life, he felt unsure of himself.

Kinnock knew he was young and inexperienced for the job. He could be the first person since Ramsay MacDonald in 1924 to become prime minister without having served in a government. When Hugo Young, the late Guardian commentator, wrote that he had “a pass degree in industrial relations from Swansea University on the second try in a bad year”, the sneer, according to Hattersley, “worried Neil much more than it should have done”.

References in Kinnock’s papers suggest that Hattersley is right. “Hugo Young was a friend of mine, and I told him that this was an appalling and stupid thing to write,” Hattersley says. “Neil could hold his own intellectually with anyone he might need to, as Labour leader or as prime minister.”

Young was an Oxford man, like every prime minister since 1945 except Churchill and Callaghan (and later Major), who did not go to university. The snobbery should not have upset Kinnock, but, as Hattersley recalls, it did.

Unfair attacks, whether from Norman Tebbit on the right, Tony Benn on the left, or Hugo Young on the higher intellectual plane, wounded him. As Denis Healey once said to Hattersley: “It’s all right for us. We’ve been up to our eyes in shit for years. He’s not used to it.”

Kinnock and Hattersley had fought each other for the leadership, but, unlike Blair and Brown a generation later, they buried their personal differences in the common cause. “Neil and I got on,” Hattersley wrote in his autobiography Who Goes Home?, adding rather archly:

We were never soulmates. Perhaps we were not even “pals” – Neil’s definition of the proper relationship between leader and deputy. But we only grumbled about each other in private and went to extraordinary lengths to demonstrate mutual respect.

The 1984-85 miners’ strike was probably the worst 12 months of Kinnock’s life; nothing hurt so much as the pain inflicted when Arthur Scargill persuaded some of the mineworkers that Kinnock, the son and grandson of Welsh miners, had betrayed them. He was sure Scargill was leading them to disaster, and as he told me afterwards: “I still curse myself for not taking the chance and saying, to a miners’ meeting – I would not have said it to anyone else – ‘You will not get sympathetic action without a ballot, and coal stocks are piled up.’ ” He says now: “Conducted in a different way, it could have had a different outcome.”

There was then – and still is, even now, though they are getting rather long in the tooth – a collection of keepers of the pure Scargellite faith who were young men and women in the 1970s. Twenty-five years after the strike, I watched these grim folk, hugging their ideological correctness to themselves like a comfort blanket, cheer wildly as Scargill addressed a London rally and said: “If Neil Kinnock had have called on the whole working class to support us, Thatcher would have been out in a year.” They must have known it was nonsense.

The 1987 general election resulted in another big defeat for Labour. The wounds from the media campaign, from Kinnock’s long battle with the Bennites, and especially those from the miners’ strike, went deep. The ebullience that came naturally to him in 1983 was starting to look forced by 1987.

On Clarke’s advice, Kinnock decided he must get rid of cherished policies: unilateral nuclear disarmament, public ownership of key industries, the repeal of all of Thatcher’s anti-trade-union laws.

Yet, despite the relentless rebranding and repackaging, his advisers insisted that there were still vestiges of the old Kinnock to be suppressed. Mandelson wrote privately to Hattersley that Kinnock’s “values and rhetoric are still tied strongly to the ‘have-nots’. He cares too much. He’s too much of a socialist and hates the idea of being seen as anything different.”

Labour’s then pollster Bob Worcester of MORI looked at the evidence from his research and said there was a different problem. Kinnock, he said, must go back to being Neil. The statesmanlike waffling was not going down well. But the skids were under Worcester, because his advice too often conflicted with that of the praetorian guard. They had forced Kinnock to give up the simple, direct, passionate language that won hearts. He was now delivering his thoughts in shapeless bundles of words.

Kinnock’s enemies, and perhaps some of his friends, had done something much more terrible to him than he did to them. They made him, just occasionally, boring.

Yet Labour was still polling better than the Conservatives. Its by-election results were improving. Its results in the European elections of 1989 were good. The Bennites were in full retreat. The poll tax, introduced first in Scotland in 1989 and then in England and Wales in 1990, was hugely unpopular. But Labour’s internal feuds were poisonous and well publicised, and Kinnock’s personal poll ratings stubbornly refused to improve. Near the end of 1990 the Conservative Party finally dumped Thatcher and elected John Major to succeed her. Major hastened to get rid of the poll tax and Labour’s climb in the polls halted.

Eight days before polling day in the 1992 general election came the Sheffield rally. It was supposed to be, as an internal memorandum put it, “a TV spectacular . . . full of music, movement of people, light effects . . . As the main BBC News comes on air, we will set off the indoor pyrotechnics, light shows and confetti guns . . .”

Earlier speakers overran, so Kinnock appeared on the platform late, and as he did so the 10,000 keyed-up people at the Sheffield Arena erupted. His repeated “We’re awright!” went down a treat in the hall – and came across dreadfully on television. But the problem was not Kinnock’s performance: it was the gaudy triumphalism of the whole affair. And if the praetorian guard had not kept the lid so firmly closed on Neil Kinnock, the dam inside him might not have burst on that Sheffield stage.

The election lost, Kinnock departed with haste and dignity and in 1995 became one of Britain’s two European commissioners. A decade ago he left the commission, took a peerage, and seems to have got his bounce back. The scion of miners had carried the heavy burden of being Scargill’s scapegoat for three decades. But the last time I met him, just a few weeks ago, we were coincidentally both at the Hampstead Theatre in London to watch Wonderland, Beth Steel’s fine play about the miners’ strike. Kinnock couldn’t contain his delight.

“The whole thing was brilliant,” he said. “And all the more amazing because Beth Steel was a baby when the strike took place, though her dad was a pit deputy – he only retired two weeks ago. The production, acting, staging was all great and the story was told with accuracy and passion. She’s doing banking next. Can’t wait!”

A few days later he was still enthusing, but less excitably: “It had a core of passionate dispassion. She was committed to pursuing a truth about the times from the perspective of the coalfield. She conveyed the sense of hopelessness overcome by determination to deal with adversity.”

Charles Clarke still says Kinnock was a great Labour leader – but the sting in the tail is that he does so in order to draw an unfavourable comparison with Ed Miliband, who he predicts will not win next year’s general election. “I think the most likely outcome is a Tory overall majority,” Clarke told the Huffington Post. “Neil has far, far more qualities than Ed Miliband as a leader. Neil was a fantastic leader and brought Labour back towards victory.”

Kinnock disagrees vehemently: “Ed is very bright, gutsy, and he takes on the big targets and presses them with intelligence and courage. Look at what he’s taken on: the press, the banks, the energy companies; he stopped us from becoming engaged in Syria [with the House of Commons vote on intervention in 2013]. Without his insight and fortitude, Cameron would have got his majority for that.

“He’s the opposite of David Cameron, who is superficial.”

And the opposite of Tony Blair?

“Very different from Tony Blair,” Kinnock says, quickly and seamlessly. All those years of not answering questions taught him a few lasting lessons.

But Hattersley spells out what Kinnock is careful about: “I think Neil felt it was time we had a leader who was real Labour.”

Hattersley says the old Kinnock is back, with all the bounce – and he is not sure it ever quite went away. “You watch him going round the Labour party conference this year,” he urges. “He knows he made a huge contribution; he knows that history will show he saved the Labour Party.

“He’s very content.”