‘Some of us are beginning to think it is the end of the project.” That was how a senior European social democrat spoke to me of the future of mainstream socialism last week. The German SPD’s abject failure in Sunday’s election will have done little to lift the gloom. After 12 years of mostly playing sidekick to Angela Merkel, it will go into opposition again, bereft of a strategy and rightly worried about the breakthrough of the far right Alternative for Germany (AfD).

If the leaders of German social democracy are feeling responsible for their own collapse and the far-right’s gain, they can at least take comfort that they are not alone. The French socialist party evaporated in the run-up to this year’s presidential election; the Dutch Labour party saw its vote slump to 5.7%; and the Austrian socialist party is facing defeat in next month’s election – which will likely bring to power the first coalition of mainstream conservatives and neo-fascists in the EU.

The accepted diagnosis among European centre-left parties is that they have either been too long in grand coalitions or become bland technocrats, easily outflanked by colourful populists of the left and right. But the problem goes deeper. The neoliberal economic model, which social democracy sought to soften and humanise, no longer works. It is on a global life-support system consisting of $12tn of central bank money.

As William Davies, a reader in political economy at Goldsmiths, University of London, writes, after 2008, neoliberalism became “in a literal sense unjustified … a ritual to be repeated, not a judgment to be believed”. It survived through a series of arbitrary state actions for which there was no supporting ideology. Since then, the ordinary people of Europe have watched themselves get poorer, their children’s future become highly uncertain and an elite centred on finance and property development get very rich. The logic mainstream parties are trying to operate no longer makes sense. Human brains are screaming out for coherent answers and, to some, economic nationalism and xenophobia look more coherent than what is on offer from the centre.

Until the centre-left learns to break with the logic of neoliberalism, and to construct an economic model that subordinates market forces to human needs, it will continue failing. The task is not to remedy or tweak the neoliberal economic model but replace it – just as fundamentally as Thatcher, Reagan and Berlusconi did in the economic counter-revolutions of the 80s and 90s.

The starting point is to stop characterising the small but vibrant leftist parties as “populist”, or “just as bad as the right”. Instead, social democrats need to learn from the radical left, and engage with them both ideologically and tactically. The Portuguese governing coalition – of the socialists and the Left Bloc – has revived the welfare state with an injection of cash, unfreezing pensions, raising benefits for families and disabled people, and boosting youth employment. Syriza in Greece has eclipsed the traditional socialist party, Pasok, not just through its moment of heroic defiance of the European Central Bank and International Monetary Fund in 2015 – but by displaying competence in government and relative immunity to the deep corruption that pervades the rest of politics. In Ireland, Sinn Féin, together with six TDs (MPs) from a far-left coalition, has become a way more powerful voice for social justice than the pallid Labour party.

Only one traditional party of European social democracy has begun the necessary transformation, and that is Labour in the UK. This year’s party conference has turned the narrow streets of Brighton into a continuous milling discussion club about the politics and economics of modern socialism. Pubs, street corners, cafes, the endless queues for fringe meetings and, of course, the beach have become overcrowded with enthusiastic, educated young Labour people preparing for the radical transformation of Britain. Some of them have the labour movement in their DNA – many others would, in Germany, be just as happy inside Die Linke or the Green party; or in Italy inside the Five Star Movement. Almost none of them would be seen dead inside Greece’s Pasok.

Sure, there are refuseniks. The former Labour minister Chris Leslie triggered guffaws across half the pubs in Brighton when his claim that “Marxism has no place in a modern Labour party” was tweeted from a grim-looking fringe event. Marxism is everywhere in the modern Labour party. Its members queued around the block to hear the Marxist anthropologist David Harvey on Sunday night. More pervasively, a gradualist form of parliamentary Marxism based around the ideas of Antonio Gramsci, as popularised by the late sociologist Stuart Hall, is the default political ideology of many Labour members under the age of 40. It forms the ideological essence of the radical social democracy that Corbynism has created.

The Labour MP Clive Lewis drew cheers at another fringe meeting when he slammed the Labour party’s sectarianism towards Caroline Lucas and the Greens. Many Labour people, including myself, want to see a strategic alliance of Labour, the progressive nationalists and the Greens in place, even if Labour were to win an overall majority. The scale of the transformation needed in Europe will likewise demand the strongest possible progressive coalitions of mainstream and radical socialist parties, progressive nationalism and left-leaning Greens.

There is no straight read-off from Labour’s revival to the revival of European social democracy. There is huge institutional and personal inertia: for if Jeremy Corbyn and the Portuguese PM Antonio Costa are right, then thousands of PhDs are wrong and so are hundreds of centre-left thinktank reports.

There is also the straitjacket of the EU. The precondition for liberating Labour from neoliberalism was that it could always think outside the box of the Lisbon treaty, which codified neoliberal doctrine into unchangeable laws. Britain’s opt-outs, from the euro and the Maastricht deficit criteria, allowed Labour politicians to ask first, what is necessary, not what is permitted under Lisbon – a mindset strengthened by Brexit.

As they start work this morning to rebuild, the first task of German social democrats is to scrap the Lisbon treaty inside their heads. Amending it in practice will be easy once they can imagine a future in which the state defends the people and the planet, not the financial elite.

• This article was amended on 9 October 2017 to clarify that the SPD spent most but not all of the past 12 years in coalition with Angela Merkel’s party.

