The general secretary of the Communist party of Britain and the leader of the Conservative party have found something they agree on. Robert Griffiths and David Cameron both think that it would be a smashing idea for Labour to choose Jeremy Corbyn as its next leader. To the mounting alarm of senior Labour figures who would like to see their party back in government some time before they die, a significant element of Labour’s membership seems to agree.

I am very wary of mysterious “private polling”, which allegedly puts the hirsute Islingtonian ahead in the race for first preferences, but there’s no dispute that he is doing much better than anyone anticipated when he threw his Lenin cap into the ring. A month ago, he was the 100-1 token candidate of the hard left who scraped into the contest only thanks to charity nominations from Labour MPs who lent him a signature (to the subsequent bitter regret of some of them) in the supposed interest of “broadening the debate”. Now the bookies have slashed his chances of winning to 5-1 and a palpable ripple of fear is running down the spine of the shadow cabinet as he wins warm applause at Labour hustings.

The interesting question is why. The hard left’s high-water mark in Labour elections was in the early 1980s when Tony Benn came within an eyebrow of beating Denis Healey in a deputy leadership contest. Since then, it has been the role of the hard left candidate to get thumped. In the contest after the 2010 defeat, Diane Abbott was their sacrificial candidate. She came last, a very poor fifth. At the outset, Mr Corbyn seemed to think that he was there to sing the old tunes – it was “my turn”, he said – before being ritually humiliated when the results came in. Whatever happens, he is going to do a lot better than that.

It is not entirely surprising that he has the endorsement of the Trotskyites and other flavours of the hard leftists who control the executive of Unite, Britain’s largest trade union. The shock to senior Labour figures is that he has attracted almost as many nominations from constituency parties as Andy Burnham and garnered more backing than Yvette Cooper. Supporters of the former health secretary and the former home secretary can be heard worrying that one of them may come in behind a man who has been an eternal backbencher since he was first returned to parliament in 1983 as an enthusiastic signatory of the “suicide note” manifesto that resulted in one of Labour’s most catastrophic defeats.

This is a big contrast with what happened in the leadership contest five years ago. Then, the spirit of New Labour still animated constituency parties. They overwhelmingly supported David Miliband with their nominations and he went on to win the vote among party members. This time around, Liz Kendall, the New Labourite standard bearer, has just five nominations from constituency parties and is widely expected to be eliminated first when the votes are counted.

The obvious conclusion to draw from this change is that the composition of Labour’s membership has moved left over the past five years. There are reasons to intuit why this might be so. Ed Miliband led the party in a way that repudiated the New Labour formula for securing power. He rubbished the Blairite nostrum that elections are won on the centre ground. He often implied – and sometimes was explicit – in suggesting that New Labour’s 13 years in office were, at best, a missed opportunity; at worst, a betrayal. As one veteran of those years puts it: “What is happening now is the consequence of five years of the leader telling the members that everything we did in government was wrong.” A meme entered the Labour mind that identified its time in office with the Iraq war and overindulgence of the rich. The progressive achievements of 13 years in power – the minimum wage, equality legislation, record spending on public services, the greatest amount of redistribution from rich to poor of any period of Labour government – all that was gradually effaced. Occasionally, big figures from the New Labour era surfaced to try to defend their record, but most of them quit the scene in frustration, despair or to do something more rewarding with their lives than being branded Tories for telling their party things it didn’t want to hear.

So it would make sense if New Labourite types at the grassroots have given up on the party since 2010 to be replaced by a different kind of membership. Talking about why his local party nominated Jeremy Corbyn, one Labour MP says: “The membership has become skewed towards people who think we should be a party of protest, not a party of power.” There’s been a manifestation of this leftward drift in the maiden speeches of the new intake of Labour MPs. Few of them sound like people we would have called Blairite before it became a toxified word. Many set their ideological compasses by Edism or further to its left. There is also some anecdotal evidence from various sources that this trend has accelerated since the election defeat with new joiners to the party predominantly hailing from the left.

We ought to give some credit for the Corbyn surge to the man himself. One of the reasons he is attracting support is because he comes over as authentic. His vivid positions and beige jackets (both circa 1983 in their vintage) add to that. The Bennite agenda he offers may be a route to the electoral wilderness, but he does have the advantage of sounding as if he really believes it would be the promised land. In his company, Andy Burnham and Yvette Cooper, with their cautiously calibrated messaging, come over as uninspirational. Liz Kendall is finding a limited audience for her attempt to get Labour to face up to the scale of the hole it is in. How much more consoling it is for a wounded and angry party to instead lap up the easy applause lines about the evils of capitalism that are proffered by the MP for Islington North.

This does not mean that the shared dream of Tories and Marxists – Jeremy Corbyn becoming leader of the opposition – will materialise. But he does not have to win to have an effect; he is already shaping the contest and with it the manner of mandate that the winner will receive. The momentum he has gathered is exerting a gravitational pull on Mr Burnham and Ms Cooper. They could directly confront the Corbyn surge by reminding Labour members of the hard truths about the hard left. That Bennery came very close to destroying the party in the 1980s. That the hard left always betrays the very people it purports to care about by ensuring that Labour never has the power to do anything to help the poor and vulnerable. That would be the brave and right thing for them to do. What stops them is that it might also be a losing thing to do. Everyone expects this contest to be settled by second preferences, which incentivises Mr Burnham and Ms Cooper to go softly on the hero of the International Marxist Tendency in the hope of picking up his transfers.

So his pitch – that Labour lost because it was not left wing enough for the electorate – is going essentially unchallenged. This is, of course, what hard leftists always say after a Labour defeat. They said it after 1979. They said it again when Labour adopted most of their prospectus and then went down to an even worse defeat in 1983. They said it after 2010. They are singing the same old anthem now about 2015. If we had promised higher taxes, more welfare, more borrowing and mass nationalisation, Ed Miliband would be in Number 10.

Meanwhile, out in the real world, serious people are investigating the true reasons why Labour came 98 seats behind the Conservatives in England and Wales, and secured 2 million fewer votes than the Tories. Today, we report on the findings of a study into an illuminating segment of the electorate: lifelong Labour voters in key marginals who stuck with the party even in the dismal context of 2010, only to finally desert it in 2015. A few of these lost Labour voters scattered to Ukip, the Lib Dems and the Greens. Most of them switched to the Tories. They told the researchers that they abandoned Labour because they didn’t trust the party with the economy, tax and spending, couldn’t see Ed Miliband as prime minister and regarded Labour as a party for welfare, not a party for the average family that wanted to get on in life. Not one of these ex-Labour voters said they rejected the party because it was not promising to establish a revolutionary socialist republic.

I am tempted to argue that Labour should go right ahead, make David Cameron’s day, choose Jeremy Corbyn and field him as its leader at the next election, so that the thesis that Labour loses because it isn’t left wing enough is finally tested to the destruction that it so richly deserves. Except that once he had been buried under a Tory landslide, I suspect that the fantasist tendency would still somehow manage to muster an argument that Lenin-cap Labour had failed because he was not quite left wing enough either.