It is 35 years since the “Gang of Four” broke away from Labour to form the Social Democratic Party, and the distance has given Bill Rodgers – now Lord Rogers of Quarry Bank, after the Liverpool suburb he grew up in – a certain calmness.

Conflicts with fellow Gang member David Owen, electoral disappointments, battles with the left are recalled dispassionately. But one moment from the past still seems to be very raw. Over coffee at his Highgate home, I ask him what it felt like to leave Labour.

“It’s a very demanding thing to break. . .” He has to pause to collect himself.

“It’s very difficult, it’s very painful, to leave,” he continues. “I could have stayed for ever as MP for Stockton-on-Tees. My people didn’t understand. They said ‘Bill, your seat’s safe, I’m supporting you, why do you have to leave?’ It’s incredibly painful.”

Most politicians arrive in the Commons as virtual unknowns, but Rodgers – who came in via a by-election in 1962 – came with sizeable reputation in the Labour movement. He had been general secretary of the Fabian Society and chief organiser of the Campaign for Democratic Socialism, the internal pressure group that agitated in favour of Hugh Gaitskell, the Labour leader. He went on to serve in the Labour governments of Harold Wilson and Jim Callaghan, holding a cabinet role as transport secretary under the latter.

As he himself says his “roots” were typically Labour. So why leave?

He describes a “continuous situation” of internal party conflict from 1960 onwards – to keep Labour a multilateralist party, in support of the manifesto against Conference, as the unofficial chief whip for Labour MPs who voted with the Conservatives to join what was then the European Economic Community, against the party line.

“It got to a point where I had to make the break.” The Christmas of 1980, he thought “about my father...what would he think, what would he feel?”Shortly after becoming an MP, his father, who had been a lifelong Labour voter, warned him against becoming a minister as “that’s too many compromises”.

The two decades of struggle had only intensified after 1970, the year that Rodgers believes it all began to go wrong. MPs had begun to come under pressure from their constituency parties. He remembers Alan Fitch, the Labour MP for Wigan, who was keen to vote to take Britain into Europe, but worried about the consequences for him.

“He came to me,” Rodgers remembers, “And said: ‘It’s alright, I’ve been talking to my wife – his wife was a teacher – and she says she’s okay to go back to work again if I lose my seat.”

The years of pressure and compromise had just been too much, and Rodgers decided to leave.

“I remember at my last speech [to his constituency], my people were crying, they didn’t understand why I had to leave...” He sighs. “And it was doubly painful for me.”

Once the break was made it was easier, although that was partly because he didn’t do it alone – his longterm friend and ally, Shirley Williams, made the switch, along with 27 other Labour MPs who made the leap to the new party.

But in the end, first-past-the-post saved Labour – and destroyed the SDP. Though the new party finished just two percentage points behind the old, Labour secured 209 seats, the SDP just six. Despite living on in myth as the cause of the landslide in 1983, the reality was in some ways sadder - the SDP took roughly equal numbers of votes from both parties. It's only real impact, as Ivor Crewe and Anthony King note in their study of the breakway, Birth, Life and Death of the SDP, was in the destruction of the political careers of those who embarked on the venture. But Rodgers sees it differently.

"You may say: why not stay and fight? [Roy] Hattersley stayed, he didn't achieve anything! He became deputy leader, and quite a good journalist, but [in politics] he achieved nothing." Rodgers believes that, had the Gang of Four not left, others in Labour would not have turned it around. Roy Hattersley only began to fight back against the party's left once the Gang of Four had left. It also, he believes, gave his old party a model it could emulate.

"[Neil] Kinnock saw how you could do it," Rodgers says, "and he helped to turn it around but then he lost the election that he very much thought he’d win [in 1992]."

Unlike many, Rodgers is not of the opinion that John Smith could have won in 1997. "I liked John Smith, he was very much a social democrat but I don't think he'd have pulled the Labour party through. It was very much Blair who did that."

Blair, Rodgers says, had many of the qualities of Gaitskell, which led many of the "younger ones" to return to the Labour party. I wonder why he didn't make the journey himself.

"It was a case of: changing once, but not changing a second time as it were," he says, quietly. The pain of leaving still hasn't gone away.

Though Rodgers was pivotal in bringing about the merger between the SDP and the Liberal party to form the Liberal Democrats, there is still an SDP leaflet in pride of place on his mantelpiece in the living room.

“I’m now a social democrat and always have been. You’ve got to allow for changes [in society] but I would say my values, my purposes are the same as they were when I joined politics as a 16-year-old.” The important thing in politics, he says, is to be "in favour of the underdog".

I wonder if he thinks that Labour’s social democrats should once again try to go it alone. “I’m sorry to see my old party the way it is,” he says. However, he is sceptical of its current generation’s ability to follow the Gang of Four’s path. “What we have here is not a leadership group [on the right of the Labour party] . . . there is no one clear leader or even a group of leaders,” he says. “There are rising stars but it takes to time to see: are they going to be consistent? Are they going to take risks? Can they be attacked, and abused, and have the necessary ability to take it?”

As for their internal opponent, “Corbyn is not a fool. We saw that in his campaign to win [the leadership] – he was straightforward, he knew what he thought. Whereas someone like [Andy] Burnham – he was just doing whatever he could to win support. They need a credible alternative – and the deputy leader of the Labour party, he’s not a credible alternative at all.”

But Labour, he thinks, will survive. “It was not a mistake[to leave], but my judgement in 1981 was that the Labour party was collapsing and that we would succeed as a social democratic or democratic socialist group. But in fact the Labour party is much more resilient as a party. Its roots are very deep, and they go across generations.”

The secret to his old party’s survival is what he calls “an understanding between the right or the centre and what I call the legitimate left”. He adds: “I call them the legitimate left because they are on the left but they believe in parliamentary democracy. Some say ‘soft’ but I say ‘legitimate’”.

“How it will work, how they will clear up, I don’t know, I think it’s a more messy situation than I have ever known.”