Another week, another setback for social democracy in Europe. On 18 June, Helle Thorning-Schmidt – the charismatic Danish prime minister, Neil Kinnock’s daughter-in-law and taker of selfies with Barack Obama – narrowly fell short of keeping power despite a remarkable comeback from political torpor two years ago. Her centre-left “red” bloc gained 47 per cent of the vote, against the centre-right “blue” bloc’s 51 per cent.

The result is further evidence that the global financial crisis has not tilted the arc of history to the left. It also illustrates another significant political story of our time: the squeezing of the established parties. In southern Europe, the Greek Syriza and the Spanish Podemos are mobilising against austerity; in the north, Eurosceptic parties of the right are on the rise.

In Denmark, the vote share of the populist Danish People’s Party (DPP) – the country’s equivalent of Ukip – surged to over 21 per cent. It became the second most successful party in the election, leapfrogging Venstre, the mainstream centre-right liberal party. The four parties that had dominated Danish politics for decades (Venstre, the Social Democrats, the Conservative People’s Party and the Social Liberals) gained only half of the vote between them.

Denmark’s shift to the right and the squeezing of its establishment are closely linked phenomena. They explain how the DPP managed to colonise ground on both sides of the electoral landscape. It has been nibbling the support of right-wing parties for over a decade by forcing immigration into the mainstream political debate. And yet, focusing on immigration alone, it struggled to get much more than 13 per cent of the vote. What made its campaign different this time was that it managed to reinvent itself as a party of the left – or the “left behind”.

The DPP performed particularly well in rural Jutland, the peninsula that makes up most of Denmark’s land mass, which suffers from lower growth rates than metropolitan areas. As one Danish politician explained to me, “The core dividing line in this election was between Jutland and Copenhagen.” The DPP has set itself up in opposition to the entire metropolitan political elite and won record levels of support.

As well as stealing votes from the centre right, the DPP tacked left by opposing the centre-left government’s plan to cut social benefits and pensions, a campaign based on “maintaining the Denmark you know”. It argued that the government should stop spending money on migrants and foreign aid, and should invest instead in social benefits for Danes. By linking migration to the future of the Scandinavian welfare state, the DPP managed to win support without sounding hysterical about immigration. In the televised debates, it was the two mainstream candidates – Venstre’s Lars Løkke Rasmussen and the Social Democrats’ Thorning-Schmidt – who were toughest on migrants.

Many Social Democrats, including a former minister who asked to remain anonymous, argue that the DPP was able to reinvent itself because their party had lost credibility by cutting taxes and consenting to austerity. The Social Democrats were caught in a pincer movement of their own making, hoping to win votes from Venstre by adopting a conservative economic platform and from the DPP by focusing on immigration. In the process, they lost their identity, leaving the door open for the DPP to grab a clutch of left-leaning voters.

The problem for Thorning-Schmidt was that although her party managed to defy expectations and top the poll with an impressive 26 per cent, the vote share of her putative coalition partners the Social Liberals and the Socialist People’s Party collapsed to under 5 per cent. Her campaign was successful for the Social Democrats but it destroyed support for her allies. It was a classic pyrrhic victory. Sweden is now the only country in Scandinavia that still has a centre-left government, contrary to the general view of the region.

Denmark’s Social Democrats were caught in the same trap as other leftist parties across the continent, unable to own or to challenge the neoliberal consensus. As a result, they were torn between reassuring the electorate by mimicking the right and mobilising it by offering a distinct alternative.

This is also the choice that will confront the Labour Party’s leadership candidates in the UK. The Danish election has relevance beyond Copenhagen. Like the human drama that captivated the world in the television series Borgen, it points to a universal progressive dilemma.