After the exhilarating shock of the 2017 general election my mind went back to September of the previous year, when I was in Liverpool at the Labour Party conference. My hotel window, high above the conference venue, looked out one way over the Mersey, where the busy dock once was, towards the Georgian splendour of Birkenhead’s Hamilton Square, and the other way towards the grand riverside public buildings of Liverpool. There was so much of the Labour Party there, in that view, and so much that matters now.

My first encounter with Labour politics took place in Birkenhead. I was the political assistant to Frank Field, who had just, as I joined him, been deselected by the Trotskyist cell Socialist Organiser. This was 1989 and the unrealistic politics and pickled dogmas of Liverpool City Council were not long in the memory. The politics of the hard left were ugly, masculine and aggressive. It was a hard schooling and nobody who had even such a small walk-on part in curtailing the hard left will ever welcome them as custodians of the party. I know which side Jeremy Corbyn was on. As I walked around the Labour conference last September, I bumped into Paul Davies, the man who Socialist Organiser tried to make the MP for Birkenhead in place of Frank. Davies was duly expelled by Neil Kinnock. He’s back now. They are all back now.

The big event in Birkenhead of my time working for Frank was the mooted closure of the Cammell Laird shipyard. Frank spent months trying to help secure a buyer for the yard. His work ensured a stay of execution but the shipyard in its old form closed in 1993. It was part of an industrial pattern that changed the face of England, and the nature of politics. As far back as 1960, in Must Labour Lose?, Richard Rose and Mark Abrams noted that Labour had to reflect the changing social composition of the nation if it was ever again to form a government. The same thing happened 30 years on. The initial Labour response to the industrial decline of the 1980s was impotent resistance. The response that crystallised into New Labour, which Frank saw and valued early, was to revive Birkenhead in a new way.

To some extent it worked. Albert Dock in Liverpool is a shiny testament to regeneration and the city is thriving. Frank has always been clear, though, that the people of Birkenhead did not share in the urban prosperity. They were worried about poor quality work, stagnant wages and immigration; all the issues that led to Britain’s departure from the European Union and, to some extent, explain the appeal of a dissenter leader such as Corbyn.

The Labour Party conference was a despondent event. The conventional analysis, upheld by dire opinion poll ratings, showed that Corbyn’s left-wing Labour offer would be rejected by the electorate. Not many of the usual corporate interests had bothered turning up and the whole saga had a perfunctory air. It felt like a party in its dying stages.

Down the road, at Momentum’s The World Transformed festival, I got the first sensation of what would subsequently happen in June 2017. There was energy and curiosity and idealism, and a fervent belief that politics mattered and could change the world. The Momentum festival was not populated by the Birkenhead Trots. It was full of intrigued young men and women who had been drawn to politics. I wrote in the New Statesman at the time that if the Labour right disparaged and disdained these people, they would pay a price.

I did not realise how soon, and I did not realise how heavy. Corbyn’s fine performance during the election campaign and his ability to gather 40 per cent of the electorate to the Labour standard threatens, even if it does not shatter, a central political assumption of the Labour right.

That assumption is given its most eloquent expression in A Strange Eventful History, a masterpiece of political history written by Frank’s predecessor as the MP for Birkenhead, Edmund Dell. The book’s thesis is that repeated attempts to create democratic socialism have always foundered because the democracy would never countenance the socialism. Dell goes through the gradual and inexorable disappearance of socialist ideas as they collides with reality. Slowly, the Labour Party creeps reluctantly to a position in which it accepts the market economy but seeks to regulate it for public good. The political counterpart of that historical case was that Labour’s conscience was salved by its left but it could only win power if it was controlled by the right.

That thesis has not quite fallen over yet. Labour did lose on 8 June, after all, and if we expunge Tony Blair from the party’s history, as many on the left feel is appropriate, Labour has not won an election for 43 years and has not won a working majority for over half a century. Corbyn, though, has got close enough to make one wonder whether, in the midst of a tumult, it is not now possible to win from the left.

Corbyn has hold of a commodity that is very precious, especially on the left: hope. The energy of optimism is all on his side. To veteran students of politics, accustomed to this rather shambolic man and his association with dubious causes, the transformation of Corbyn is astonishing but it is a phenomenon all the same that is changing the usual rules. The relevant distinction is not between Old and New Labour, it is between old and young Labour. Unless the right of the Labour Party can become poetic and elevated again, it will continue to wither, and will deserve to. The key to Corbyn’s success is not that he is “left-wing”. That is the wrong dimension. He did not really shift from right to left, he shifted from down to up. The 2017 election campaign was a downbeat Theresa May against a relentlessly optimistic Jeremy Corbyn. He has all the energy of the Momentum festival and none of the misery of the Liverpool conference.

Read more: Simon Fletcher on how the new socialism won the argument

There is still a lot to do. Labour is 58 seats behind the Conservatives and its coalition of voters is unstable. At least some of them voted Labour on the assumption that Corbyn would not become prime minister, rather than on the hope that he would. If he fights a campaign as a genuine contender, it may, strangely, prove harder for him. Whether he can bring even more young people into the political fold is open to question, but perhaps he can. The rules that have governed politics since Mrs Thatcher are changing. If the beneficiary turns out to be a shambling backbencher of 30 years standing and no great distinction, that would be the strangest and most eventful twist in all the long history of the Labour Party.

Philip Collins writes for the Times