The skirmishes since Jeremy Corbyn’s remarkable victory have avoided the main issue. The Labour Party has been sucked into debates about the rights and wrongs of serving in the shadow cabinet, the wearing of red poppies, the style of Prime Ministers’ Questions and the singing of the national anthem. Recollections of the battles of the 1980s (which I analysed at the time as political editor of the New Statesman) have prompted arguments about whether the best way to secure a progressive government in 2020 is for Labour now to split or to stay together.

There is, however, a more fundamental question that needs to be confronted head-on. It concerns the very purpose of Labour’s existence. Corbyn’s election has opened up a doctrinal chasm on the left. Can it now be bridged or not?

If it can, then Labour might fray at the edges but not shatter. But if the divide is simply too wide, and if Corbyn is still in place in two or three years’ time, then his opponents will face a stark choice: accept that Labour has reverted to an older, firmly anti-capitalist version of its purpose – or leave this party and start a new one.

Here lies the true significance of the Corbyn insurgency. It clarifies and polarises the debate that should be held about what Labour really stands for. Of course, doctrinal arguments have been held throughout the party’s history. Labour has debated the character of socialism for well over a hundred years. But, until now, the outcome has repeatedly been a fix, a fudge, disdain by the party leader or the application of machine politics to keep out the far left. In every one of its four periods of majority government since 1945, Labour has in practice come to terms with capitalism. Now, for the first time, the far left has taken over. Corbyn has already demanded nationalised railways, energy companies and banks.

Perhaps that is all; perhaps he privately embraces the market system in the rest of Britain’s economy. However, his latest plans for corporate taxes suggest no such enthusiasm. Indeed, all the evidence points in the opposite direction. In his 32 years as an MP (and in the years before that when I listened to him backing Tony Benn, Militant and other assorted Trotskyists when we were both members of Labour’s general committee in Hornsey and Wood Green, in north London), I have never come across anything he has said or written that displays any p­assion for the process of wealth creation that flows from competition among privately owned businesses.

Indeed, the opposite is the case. In November 2013, Corbyn published a column in the Morning Star headlined “Challenging capitalism”. He wrote: “It’s high time to move public ownership firmly to the centre of the political agenda.” More broadly, he has been reported as telling his Islington North Labour Party that: “Our job is not to reform capitalism; it’s to overthrow it.” No wonder he has appointed a shadow chancellor whose Who’s Who entry declares his ambition as “fermenting the overthrow of capitalism”.

In the short term, Corbyn will doubtless compromise on his policy agenda, in order to prevent an immediate revolt by more moderate Labour MPs. We should not be fooled. He is a principled socialist. His long-term aims remain. He is a leopard whose spots have never changed, and never will. In a way, that is to Corbyn’s credit. Throughout his political life he has held to a particular view of how to achieve prosperity. He thinks the best way to build a good society is for workers and elected politicians, not company shareholders, to take the big decisions in the business world.

However, that is not remotely what most of Labour’s other leading MPs want. They believe in capitalism. They do not regard it as an evil to be fought at every turn, or even as a regrettable necessity to be endured for the time being. They like its dynamism. They regard it as the best way to invent, develop and supply most goods and services. They have no wish to replace it, even as a long-term objective. They think that one of the basic ambitions of progressive government is to find the best way to encourage private-sector success and, through the judicious use of support, regulation and taxation, to harness that success to the wider task of building a fairer, better society.

Not that many of them would put it as bluntly as that. Look at the words written and spoken by Corbyn’s three opponents and, with the partial exception of Liz Kendall, you will find no celebration of the success and virtues of capitalism and the market system, merely a guarded acknowledgement of its existence. They talk about capitalism not in the manner of a sister to be embraced, but as an awkward cousin to be tolerated.

The outcome has been an ideologically lopsided debate in the leadership contest. For those who view the New Labour years as a model to be admired not reviled, it has come across as a choice between Corbyn who has been wrong but clear, and his rivals who have been right but mealy-mouthed.

This brings us to the heart of the matter. For the character of the century-long tussle between traditional socialism and working-with-capitalism social democracy has always been thus, as left-wing clarity vies with centrist mush. The process has been consistently messy, and frequently frustrating; but it has also been seldom catastrophic and occasionally triumphant. Understanding the evasive culture of Labour’s internal discourse through the 20th century helps us to see why Corbyn’s election could mark such a profound moment in the party’s history.

***

Morgan Phillips, Labour’s general secretary in the 1950s and one of the old school of machine politicians, made the important observation: “The Labour Party owes more to Methodism than to Marxism.” This is far more than a neat contrast between two words beginning with “M”. It reflects a profound truth about the way Labour has evolved. When Keir Hardie arrived in the House of Commons in 1892 and railed against poverty and exploitation, he couched his argument in moral terms. In his maiden speech in 1893 he spoke not of Karl Marx or class war, but “the horrors of sweating, of low wages, of long hours, and of deaths from starvation”. His proposals – in that particular case, to curb cheap imports that cost British workers their jobs – were rooted in ethical concern rather than ideological conviction.

That set the course for the decades that followed. Even the famous, or notorious, Clause Four, agreed in 1918, fits the pattern. It was crafted with care by Sidney Webb, the most prominent of the early Fabians. In its final form (it went through various drafts over a period of months), it stated that Labour’s objective was:

To secure for the workers by hand or by brain the full fruits of their industry and the most equitable distribution thereof that may be possible upon the basis of the common ownership of the means of production, distribution and exchange, and the best obtainable system of popular administration and control of each industry or service.

This is generally regarded as a call for ­full-scale nationalisation. But “common ownership” is a far looser term, and the phrase “as may be possible” suggests an incremental rather than revolutionary process. This was deliberate. Overshadowing the Clause Four debate was the Russian Revolution. It inspired some in the west but terrified many more. Webb and his colleagues were determined to distance Labour from the Soviet model. In October 1917, days before Lenin finally seized control, Webb wrote in the Observer:

It [Clause Four] is a socialism which is no more specific than a definite repudiation of the individualism that characterised all the political parties of the past generations . . . This declaration of the Labour Party leaves it open to choose from time to time whatever forms of common ownership, from the co-operative store to the nationalised railway, and whatever forms of popular administration and control of industry, from national guild to ministries of employment, and municipal management may, in particular cases, commend themselves.

In the context of its time, with Britain engaged in the Great War and with much of the economy under state control, as well as Russia turning communist, Webb’s ambition was modest, even insipid.

That said, Labour’s election manifestos in the 1920s and 1930s preached a more muscular socialism. (In 1931 the party proclaimed that “the decay of capitalist civilisation brooks no delay”.) But the party’s two short spells of minority government, in 1924 and 1929-31, gave it little chance to turn words into action. Its first proper test came in 1945, with Clement Attlee’s landslide victory.

Attlee wanted to fight the election with no specific commitments to nationalisation. But in December 1944 the party conference defied his wishes and voted overwhelmingly for “the transfer to public ownership of the land, large-scale building, heavy industry and all forms of banking, transport and fuel and power”. Attlee blithely ignored most of this list. True, his government nationalised the mines and the railways; but given how badly these had been run before the war, one could make a perfectly pragmatic, non-ideological case for taking them over. By 1949, Harold Wilson, president of the Board of Trade, was proclaiming that he had made “a bonfire of controls” to release the energies of the private sector.

As the postwar years ushered in the consumer society, Clause Four looked increasing out of place. What was the relevance of “common ownership” to a world of privately owned homes, cars, television sets and washing machines? In 1959, a few weeks after Labour’s third successive election defeat, the party’s leader, Hugh Gaitskell, sought to change it.

Once again, the party leader argued for pragmatism rather than explicitly for the virtues of capitalism: nationalisation, he said, was one of a number of means for pursuing freedom, social justice and the public interest. Once again, the leader was opposed by left-wing calls for state socialism. Frank Cousins, general secretary of the Transport and General Workers’ Union, the biggest trade union in Britain, addressed the party conference in terms that could have come straight from the Corbyn playbook:



“Let us give over pretending we have to get half a million Tory people to change their allegiance at voting time. There are five million or six million people who are socialists in embryo waiting for us to go out and harness them to the power machine we want to drive.”

Once again, as in 1944, the party leader was defeated. But once again, when Labour was next in power (under Wilson, elected party leader after Gaitskell’s death), it disregarded the conference decision. Clause Four lived on, yet as a symbol rather than a strategy. Only in the catastrophic election of 1983 did Labour take it seriously.

Finally, in 1995, Tony Blair did persuade the National Executive Committee and a Labour conference to agree a new Clause Four:

The Labour Party is a democratic socialist party. It believes that by the strength of our common endeavour, we achieve more than we achieve alone, so as to create for each of us the means to realise our true potential and for all of us a community in which power, wealth and opportunity are in the hands of the many not the few; where the rights we enjoy reflect the duties we owe, and where we live together freely, in a spirit of solidarity, tolerance and respect.

Out went the vague ambition of “common ownership”. In came a perfectly sensible statement of the ethic of co-operation, but nothing that made the case for any kind of economic freedom, let alone full-blown market capitalism. Blair can claim the credit for refusing to take the Attlee/Gaitskell/Wilson route of ignoring Clause Four and disregarding party conference decisions. But he did not win the argument for a pro-capitalist version of social democracy, because he never spelled it out. He implemented policies that the left now attacks as “market liberalism” not by persuading his party of its virtues but by winning elections and asserting his authority.

***

Thinking with the wisdom of hindsight, we should not be surprised that the anti-capitalist left has revived. The hard truth is that it was never defeated because it was never properly engaged. It was simply thrust to the margins, where it bided its time. After two general election defeats, the left appeals to party activists in a way it could never do during the era of Blair’s election victories. And the character of the recent leadership contest matches the character of every significant doctrinal contest through the Labour Party’s history, with Corbyn arguing his case with clarity and his opponents ducking and weaving.

The difference is that Labour now has a leader, for the first time since at least the Second World War, who actually believes in the policies that the left has consistently advocated and previous leaders equally consistently ignored.

Could things have worked out differently? Could Labour done more than hold the left at bay: could it have won a head-on doctrinal battle?

Perhaps. Such a battle was waged, and won, more than 50 years ago in Germany. In 1959 the German Social Democratic Party (SPD) – Labour’s sister party – met at Bad Godesberg and agreed a new doctrine. In the extract here, the final sentence is the one most frequently quoted, but the whole paragraph is striking, if only because no British Labour conference has ever agreed anything remotely like it:

Free choice of consumer goods and services, free choice of working place, freedom for employers to exercise their initiative as well as free competition are essential conditions of a Social Democratic economic policy. The autonomy of trade unions and employers’ associations in collective bargaining is an important feature of a free society. Totalitarian control of the economy destroys freedom. The Social Democratic Party therefore favours a free market wherever free competition really exists. Where a market is dominated by individuals or groups, however, all manner of steps must be taken to protect freedom in the economic sphere. As much competition as possible – as much planning as necessary.



How come the SPD so long ago confronted left-wing socialism in a way that even Blair at the height of his popularity never attempted? The immediate context plainly played a part. The SPD had lost every postwar election and knew it had to change. Across the border, East Germany, and the Soviet bloc generally, were giving Marxist notions a bad name. Nothing that sniffed of communism, in however dilute a form, was likely to be popular in West Germany.

But there was something else. There was a fundamental difference between Labour’s history and that of the SPD. As we have seen, Labour, with its Methodist-not-Marxist roots, has always been a party of ethics rather than ideology. In contrast, the SPD was created in 1863 as an explicitly Marxist party. That is not to say Marx was a fan. In 1875 the SPD adopted a programme that he strongly criticised as too concerned with formal economic structures and too little with the dynamic of class struggle. However, for the following eight decades, the SPD viewed the world through the prism of ideology.

The essence of what happened in the years leading to Bad Godesberg was that the realisation grew within the SPD that its ideological theory was wrong. State control of the economy was a bad idea. A competitive market economy was intrinsically superior. Governments should intervene only when markets failed.

In a way, the SPD in the 1950s applied the tenets of the Enlightenment to itself. It approached its problems empirically. It pondered the evidence and concluded that Marxist socialism did not work, while properly regulated market capitalism did.

Labour has never engaged in any such Enlightenment-style debate. This is because the advocates of left-wing socialism inside Labour (leaving side the Trotskyists, communists and fellow-travellers who have occupied its fringes from time to time) have argued from a moral rather than a theoretical standpoint. And the ethic of co-operation and fairness does not lend itself easily to empirical investigation.

Thus Labour finds itself with a new leader who rejects the accommodation with market capitalism that every Labour leader since the Second World War, except for Michael Foot, has in practice upheld but none has properly persuaded his party to embrace.

What now? By 2020, one of three things will have happened.

1. Jeremy Corbyn will have maintained control over his party, which may have frayed but not split;

2. Corbyn will have been replaced by a more electable, less left-wing leader;

3. Labour will have split, leaving Corbyn as the leader of a significantly diminished group of MPs.

I don’t know which of these will happen, but I suspect that the outcome will depend on how many MPs decide to fight his left-wing doctrine directly. Most Labour MPs think Corbyn’s politics are bonkers. Left to their conscience, most would strive to remove him at the earliest opportunity or, if that fails, break away and start a new party. But will enough of them combine to do either of these things? Or will they recall the unsettling dictum that the plural of conscience is conspiracy, do nothing to risk being deselected as party candidates in 2020, and quietly hope that Corbyn’s leadership will crumble of its own accord?

I fear that the quiet life will win the day, that Corbyn will become entrenched, and that a head-on doctrinal dispute will, as always, be avoided. For a century, fudging the issue has occasionally allowed Labour to build an election-winning, big-tent coalition of progressive voters. Today, that approach guarantees disaster. It will leave Corbyn free to promote his electorally toxic and economically destructive brand of left-wing politics. If that is what happens, Labour’s tent will become a lot smaller and the party will cease to be fit for purpose.

Peter Kellner is the president of YouGov. Read his analysis of the new polling data that shows the challenge for Jeremy Corbyn here