Last September, Il Partito Democratico, the Italian Democratic Party, asked me to talk about politics and the internet at its summer school in Cortona. Political summer schools are usually pleasant — Cortona is a medieval Tuscan hill town with excellent restaurants — and unexciting. Academics and public intellectuals give talks organised loosely around a theme; in this case, the challenges of ‘communication and democracy’. Young party activists politely listen to our speeches while they wait to do the real business of politics, between sessions and at the evening meals.

This year was different. The Italian Democratic Party, which dominates the country’s left-of-centre politics, knew that it was in trouble. A flamboyant blogger and former comedian named Beppe Grillo had turned his celebrity into an online political force, Il Movimento 5 Stelle (the Five Star Movement), which promised to do well in the national elections. The new party didn’t have any coherent plan beyond sweeping out Old Corruption, but that was enough to bring out the crowds. The Five Star Movement was particularly good at attracting young idealists, the kind of voters who might have been Democrats a decade before.

Worries about this threat spilt over into the summer school. The relationship between communication and democracy suddenly had urgent political implications. The Democratic Party had spent two decades suffering under the former prime minister Silvio Berlusconi’s stranglehold on traditional media. Now it found itself challenged on the left too, by internet-fuelled populists who seemed to be sucking attention and energy away from it.

When Bersani started talking, he gave a speech that came strikingly close to a counsel of despair

The keynote speaker at the summer school, the Democratic Party leader and prospective prime minister Pier Luigi Bersani, was in a particularly awkward position. Matteo Renzi, the ‘reformist’ mayor of Florence, had recently challenged Bersani’s leadership, promising the kind of dynamism that would appeal to younger voters. If Bersani wanted to stay on as party leader, he had to win an open primary. The summer school gave him a chance to speak to the activists in training, and try to show that he was still relevant.

I was one of two speakers warming up the crowd for Bersani. The party members and reporters endured us patiently enough as they waited for the real event. However, when Bersani started talking, he gave a speech that came strikingly close to a counsel of despair. He told his audience that representative democracy, European representative democracy in particular, was in crisis. Once, it had offered the world a model for reconciling economy and society. Now it could no longer provide the concrete benefits — jobs, rights, and environmental protection — that people wanted. In Italy, Berlusconi and his allies had systematically delegitimized government and undermined public life. The relationship between politics and society was broken.

Bersani knew what he didn’t want — radical political change. Any reforms would have to be rooted in traditional solidarities. But he didn’t know what he did want either, or if he did, he wasn’t able to describe it. His speech was an attack, swathed in the usual billowing abstractions of Italian political rhetoric, on the purported radicalism of both his internal party opponent and the Five Star Movement. He didn’t really have a programme of his own. He could promise his party nothing except hard challenges and uncertain outcomes.

Why do social democrats such as Bersani find it so hard to figure out what to do? It isn’t just the Italians who are in trouble. Social democrats in other countries are also in retreat. In France, Francoise Hollande’s government has offered many things: a slight softening of austerity (France’s deficit this year will be somewhat higher than the European Commission would like); occasional outbursts of anti-business rhetoric (usually swiftly contradicted by follow-up statements); higher taxes on the very rich (to be rolled back as soon as possible). What it has not offered is anything approaching a coherent programme for change.

Germany’s Social Democrats are suffering, too. The Christian Democrat-led government can get away with austerity measures as long as it convinces voters that it will do a better job of keeping their money safe from the Spaniards, Italians and Greeks. And the Social Democratic Party’s candidate for Chancellor, Peer Steinbrück, is not well placed to object. In 2009 he helped introduce a constitutional measure to limit government spending, hoping that this would make his party look more responsible. He now appears like a weaker, less resolute version of his opponent, Chancellor Angela Merkel, and has 32 per cent job approval.

Greece’s mainstream socialist party, Pasok, won only 12.3 per cent of the vote in the election in June last year. Spain’s social democrats are perhaps in even greater disarray than the conservative government. Ireland’s Labour Party, a junior party in the current government, saw its vote collapse from 21 per cent to 4.6 per cent in a by-election in March.

Where they are in opposition, European social democrats don’t know what to offer voters. Where they are in power, they don’t know how to use it. Even in the United States, which has never had a social democratic party with national appeal, the Democrats have gradually changed from a party that belonged ambiguously to the left to one that spans the limited gamut between the ever-so-slightly-left-of-centre and the centre-right. It, too, has had enormous difficulty in spelling out a new agenda, because of internal divisions as well as entrenched hostility from the Republican Party.

This isn’t what was supposed to happen. In the 1990s and the 2000s, right-wing parties were the enthusiasts of the market, pushing for the deregulation of banks, the privatisation of core state functions and the whittling away of social protections. All of these now look to have been very bad ideas. The economic crisis should really have discredited the right, not the left. So why is it the left that is paralysed?

Colin Crouch’s disquieting little book, Post-Democracy (2005), provides one plausible answer. Crouch is a British academic who spent several years teaching at the European University Institute in Florence, where he was my academic supervisor. His book has been well read in the UK, but in continental Europe its impact has been much more remarkable. Though he was not at the Cortona summer school in person, his ideas were omnipresent. Speaker after speaker grappled with the challenge that his book threw down. The fear that he was right, that there was no palatable exit from our situation, hung over the conference like a dusty pall.

Crouch sees the history of democracy as an arc. In the beginning, ordinary people were excluded from decision-making. During the 20th century, they became increasingly able to determine their collective fate through the electoral process, building mass parties that could represent their interests in government. Prosperity and the contentment of working people went hand in hand. Business recognised limits to its power and answered to democratically legitimated government. Markets were subordinate to politics, not the other way around.

The realm of real democracy — political choices that are responsive to voters’ needs — shrinks ever further

At some point shortly after the end of the Second World War, democracy reached its apex in countries such as Britain and the US. According to Crouch, it has been declining ever since. Places such as Italy had more ambiguous histories of rise and decline, while others still, including Spain, Portugal and Greece, began the ascent much later, having only emerged from dictatorship in the 1970s. Nevertheless, all of these countries have reached the downward slope of the arc. The formal structures of democracy remain intact. People still vote. Political parties vie with each other in elections, and circulate in and out of government. Yet these acts of apparent choice have had their meaning hollowed out. The real decisions are taken elsewhere. We have become squatters in the ruins of the great democratic societies of the past.

Crouch lays some blame for this at the feet of the usual suspects. As markets globalise, businesses grow more powerful (they can relocate their activities, or threaten to relocate) and governments are weakened. Yet the real lessons of his book are about more particular forms of disconnection.

Neo-liberalism, which was supposed to replace grubby politics with efficient, market-based competition, has led not to the triumph of the free market but to the birth of new and horrid chimeras. The traditional firm, based on stable relations between employer, workers and customers, has spun itself out into a complicated and ever-shifting network of supply relationships and contractual forms. The owners remain the same but their relationship to their employees and customers is very different. For one thing, they cannot easily be held to account. As the American labour lawyer Thomas Geoghegan and others have shown, US firms have systematically divested themselves of inconvenient pension obligations to their employees, by farming them out to subsidiaries and spin-offs. Walmart has used hands-off subcontracting relationships to take advantage of unsafe working conditions in the developing world, while actively blocking efforts to improve industry safety standards until 112 garment workers died in a Bangladesh factory fire in November last year. Amazon uses subcontractors to employ warehouse employees in what can be unsafe and miserable working conditions, while minimising damage to its own brand.

Instead of clamping down on such abuses, the state has actually tried to ape these more flexible and apparently more efficient arrangements, either by putting many of its core activities out to private tender through complex contracting arrangements or by requiring its internal units to behave as if they were competing firms. As one looks from business to state and from state to business again, it is increasingly difficult to say which is which. The result is a complex web of relationships that are subject neither to market discipline nor democratic control. Businesses become entangled with the state as both customer and as regulator. States grow increasingly reliant on business, to the point where they no longer know what to do without its advice. Responsibility and accountability evanesce into an endlessly proliferating maze of contracts and subcontracts. As Crouch describes it, government is no more responsible for the delivery of services than Nike is for making the shoes that it brands. The realm of real democracy — political choices that are responsive to voters’ needs — shrinks ever further.

Politicians, meanwhile, have floated away, drifting beyond the reach of the parties that nominally chose them and the voters who elected them. They simply don’t need us as much as they used to. These days, it is far easier to ask business for money and expertise in exchange for political favours than to figure out the needs of a voting public that is increasingly fragmented and difficult to understand anyway. Both the traditional right, which always had strong connections to business, and the new left, which has woven new ties in a hurry, now rely on the private sector more than on voters or party activists. As left and right grow ever more disconnected from the public and ever closer to one another, elections become exercises in branding rather than substantive choice.

Crouch was writing Post-Democracy 10 years ago, when most people thought that things were going quite well. As long as the economy kept delivering jobs and growth, voters didn’t seem to mind about the hollowing out of democracy. Left-of-centre parties weren’t worried either: they responded to the new incentives by trying to articulate a ‘Third Way’ of market-like initiatives that could deliver broad social benefits. Crouch’s lessons have only really come home in the wake of the economic crisis.

The problem that the centre-left now faces is not that it wants to make difficult or unpopular choices. It is that no real choices remain. It is lost in the maze, able neither to reach out to its traditional bases of support (which are largely dying or alienated from it anyway) nor to propose any grand new initiatives, the state no longer having the tools to implement them. When the important decisions are all made outside of democratic politics, the centre-left can only keep going through the ritualistic motions of democracy, all the while praying for intercession.

Most left-wing parties face some version of these dilemmas. Cronyism is less a problem than an institution in the US, where decision-makers relentlessly circulate between Wall Street, K Street, and the Senate and Congress. Yet Europe has some particular bugbears of its own. Even if national political systems were by some miracle to regain their old responsiveness, the power of decision has moved to the European Union, which is dominated by a toxic combination of economic realpolitik and bureaucratic self-interest. Rich northern states are unwilling to help their southern neighbours more than is absolutely necessary; instead they press for greater austerity. The European Central Bank, which was deliberately designed to be free of democratic oversight, is becoming ever more important, and ever more political. Social democrats once looked to the EU as a bulwark against globalisation — perhaps even a model for how the international economy might be subjected to democratic control. Instead, it is turning out to be a vector of corrosion, demanding that weaker member states implement drastic economic reforms without even a pretence of consultation.

Let’s return to Italy, the laboratory of post-democracy’s most grotesque manifestations. Forza Italia, Silvio Berlusconi’s elaborate simulacrum of a political party, is a perfect exemplar of Crouch’s thesis: a thin shell of branding and mass mobilisation, with a dense core of business and political elites floating free in the vacuum within.

After the Cortona summer school, Bersani won his fight with Renzi in November last year and led his party into the general election. His coalition lost 3.5 million votes but still won the lower house in February, because the Italian electoral system gives a massive bonus to the biggest winner. It fell far short of a majority in the upper house and is doing its hapless best to form a government. Grillo’s Five Star Movement, on the other hand, did far better than anyone expected, winning a quarter of the votes. Grillo has made it clear that his party will not support the Democratic Party. Renzi has tried to advance himself again as a compromise leader who might be more acceptable to Grillo, so far without success. In all likelihood there will be a second general election in a few months.

‘We die if a movement becomes a party. Our problem is to remain a movement in parliament, which is a structure for parties. We have to keep a foot outside’

The Italian Democratic Party is caught on one tine of the post-democratic dilemma. It is trying to work within the system as it is, in the implausible hope that it can produce real change within a framework that almost seems designed to prevent such a thing. As the party has courted Grillo, it has started making noises about refusing to accept austerity politics and introducing major institutional reforms. It is unclear whether senior Democratic figures believe their new rhetoric; certainly no one else does. If the party does somehow come to power, the most it will do is tinker with the system.

The Five Star Movement has impaled itself on the other tine, as have the Indignados in Spain, Occupy in the US and UK, and the tent movement in Israel. All have gained mass support because of the problems of post-democracy. The divide between ordinary people and politicians has grown ever wider, and Italian politicians are often corrupt as well as remote. The Five Star Movement wants to reform Italy’s institutions to make them truly democratic. Yet it, too, is trapped by the system. As Grillo told the Financial Times in October: ‘We die if a movement becomes a party. Our problem is to remain a movement in parliament, which is a structure for parties. We have to keep a foot outside.’

The truth is, if the Five Star Movement wants to get its proposals for radical change through the complex Italian political system, it will need to compromise, just as other parties do. Grillo’s unwillingness even to entertain discussions with other parties that share his agenda is creating fissures within his movement. Grillo is holding out for a more radical transformation, in which Italian politics would be replaced by new forms of internet-based ‘collective intelligence’, allowing people to come together to solve problems without ugly partisan bargaining. In order to save democracy, the Five Star Movement would like to leave politics behind. It won’t work.

The problems of the Italian left are mirrored in other countries. The British Labour Party finds itself in difficulty, wavering between a Blairite Third Wayism that offers no clear alternative to the present government, and a more full-blooded social democracy that it cannot readily define. The French left has mired itself in scandal and confusion. The Greek left is divided between a social democratic party that is more profoundly compromised than its Italian equivalent and a loose coalition of radicals that wants to do anything and everything except find itself in power and be forced to take decisions.

All are embroiled, in different ways, in the perplexities of post-democracy. None has any very good way out. Ever since France’s president François Mitterrand tried to pursue an expansive social democratic agenda in the early 1980s and was brutally punished by international markets, it has been clear that social democracy will require either a partial withdrawal from the international economy, with all the costs that this entails, or a radical transformation of how the international economy works.

It is striking that the right is not hampered to nearly the same extent. Many mainstream conservatives are committed to democracy for pragmatic rather than idealistic reasons. They are quite content to see it watered down so long as markets work and social stability is maintained. Those on the further reaches of the right, such as Greece’s Golden Dawn, find it much easier than the Five Star Movement or Syriza, the Greek radical-left coalition, to think about alternatives. After all, they aren’t particularly interested in reforming moribund democratic institutions to make them better and more responsive; they just want to replace them with some version of militaristic fascism. Even if these factions are unlikely to succeed, they can still pull their countries in less democratic directions, by excluding weaker groups from political protection. The next 10 years are unlikely to be comfortable for immigrants in southern Europe.

Post-democracy is strangling the old parties of the left. They have run out of options. Perhaps all that traditional social democracy can do, to adapt a grim joke made by Crouch in a different context, is to serve as a pall-bearer at its own funeral. In contrast, a new group of actors — the Five Star Movement and other confederations of the angry, young and dispossessed — have seized a chance to win mass support. The problem is, they seem unable to turn mass frustration into the power to change things, to create a path for escape.

Perhaps, over time, they will figure out how to engage with the mundane task of slow drilling through hard boards that is everyday politics. Perhaps, too, the systems of unrule governing the world economy, gravely weakened as they are, will fail and collapse of their own accord, opening the space for a new and very different dispensation. Great changes seem unlikely until they happen; only in retrospect do they look inevitable. Yet if some reversal in the order of things is waiting to unfold, it is not apparent to us now. Post-democracy has trapped the left between two worlds, one dead, the other powerless to be born. We may be here for some time.