THE past 30 years have been dismal ones for the labour movement. In the American private sector trade-union density (ie, the proportion of workers who belong to unions) has fallen from a third in 1979 to just 7% today. In Britain it has dropped from 44% to 15%. Nor is this just an Anglo-Saxon oddity: less than a fifth of workers in the OECD belong to unions.

There is one big exception to this story of decline, however: the public sector. In the Canadian public sector union density has increased from 12% in 1960 to more than 70% today. In America it has increased over the same period from 11% to 36% (see chart). There are now more American workers in unions in the public sector (7.6m) than in the private sector (7.1m), although the private sector employs five times as many people. Union density is now higher in the public sector than it was in the private sector in its glory days, in the 1950s.

Even countries that have seen a dilution of union density in the public sector have seen it stabilise at a much higher level than in the private sector. In Britain density has fallen dramatically from 82% in 1979, but has stabilised at about 56%. Reliable global statistics are hard to come by; but evidence from many countries (including Germany and Japan) suggests that the gap between the public and private sectors is both substantial and growing.

This private-public shift has transformed the trade union movement. In the 1950s unions were solidly working class, dominated by men who had left school at 16 and leant left on economics but right on social issues. Today they are much more middle-class: more than a quarter of American unionists have college degrees, and even more have liberal views on social and environmental issues.

The shift has also created tension between the public and private sectors. The private sector is dominated by competition and turbulence. Performance-related pay is the norm, and redundancy commonplace. The public sector, by contrast, is a haven of security and stability. Many people have jobs for life and performance measures are rare. The result is a paradox: the typical public worker is better off than the people he is supposed to serve, and the gap has widened significantly over the past decade. In America, pay and benefits have grown twice as fast in the public sector as they have in the private sector.

Now that the sovereign-debt crisis is forcing governments to put their houses in order, the growing discrepancy between conditions in the public and private sectors has eroded much of the sympathy public-sector workers might once have enjoyed. This briefing will look at what the future holds for them. But first it will try to answer two questions: how did public-sector unions become so powerful? And what impact has their power had on the way the public sector works?

I'm all right, Jack

Public-sector unions are some of the world's most powerful interest groups. Many of them have large memberships and comparably large wallets: the American National Education Association, the main teachers' union, has 3.2m members, an annual budget of over $300m and a vibrant tradition of political activism. But their influence goes much deeper. In many countries unions prop up the left. In Britain Ed Miliband, the leader of the Labour Party, owes his job to trade-union votes. In America Andy Stern, the head of the Service Employees International Union, was the most frequent guest at the White House in the first six months of Barack Obama's presidency.

Public-sector unions enjoy advantages that their private-sector rivals only dream of. As providers of vital monopoly services, they can close down entire cities. And as powerful political machines, they can help to pick the people who sit on the other side of the bargaining table. Daniel DiSalvo, the author of an excellent essay on America's public-sector unions in National Affairs, points out that the American Federation of State, County and Municipal Employees was the biggest contributor to political campaigns in 1989-2004. He also notes that such influence is more decisive in local campaigns, where turnout is low, than in national ones.

Even if they fail to elect “their” candidates, public-sector unions have a relatively easy time negotiating with politicians. Private-sector bosses are accustomed to playing hardball with unions because they know they can go bankrupt if they don't. Politicians have no such discipline: they can always raise taxes or borrow from future generations. Those who have challenged the unions have often regretted it. California's former governor, Arnold Schwarzenegger, tried to fight the unions in the court of public opinion, only to be outgunned. Others have attempted a more stopgap approach, only to get the blame when services are disrupted.

Economists still debate exactly what impact public-sector unions have on pay. Evidence from the American Bureau of Labour Statistics support the conservative argument that they have used their power to extract a wage premium: public-sector workers earn, on average, a third more than their private-sector counterparts. Left-leaning economists reply that public-sector workers are, on average, better educated. Whatever the merits of this argument, three things seem clear. Unions have suppressed wage differentials in the public sector. They have extracted excellent benefits for their members. And they have protected underperforming workers from being sacked.

Wage differentials are relatively small in the public sector. Lower-level workers, such as secretaries, are usually better paid than their private-sector equivalents, whereas higher-level workers are worse paid. This not only makes it difficult to attract high-flyers into the public sector, but also makes it hard to raise standards by, for instance, putting the best head teachers in charge of groups of schools.

At the same time, benefits are generous in the public sector. Governments tend to give their workers light workloads and generous pensions in lieu of higher wages (which have to come out of the current budget). In America teachers teach for a mere 180 days a year. In Brazil they have the right to take 40 days off a year—out of 200 working days—without giving an explanation or losing a centavo of pay. The defined-benefits revolution that has swept through the private sector has hardly touched the public one: 90% of American state- and local-government workers have defined-benefit plans, compared with 20% of private-sector workers.

Generous pensions have produced an epidemic of early retirement. In Brazil civil servants can retire on full pay after 35 years on the job (30 for women) and teachers can retire after 30 years (25 for women). The result is that Brazil spends as high a proportion of its GDP on pensions (12%) as Britain does, even though the population is much younger. In Poland soldiers and policemen can retire after just 15 years, so it is possible to come across 33-year-old retirees. Add to this the fact that any public-sector worker can hide behind union power to game the system—82% of senior California Highway Patrol officers discover a disabling injury about a year before they retire—and you have a dysfunctional mess.

Unions have also made it almost impossible to sack incompetent workers. In Greece there is a law against sacking government workers solely on grounds of poor performance. In other countries there might as well be. Mary Jo McGrath, a Californian lawyer, says that “getting rid of a problem teacher can make the O.J. [Simpson] trial look like a cakewalk.” In 2000-10 the Los Angeles school district spent $3.5m trying to get rid of seven of its 33,000 teachers, and succeeded with only five. The problem extends across the country (see article).

Incompetence is so endemic that several countries have invented phrases to deal with it. Brazilians joke that public-sector workers turn up on the first day, hang their jackets on the back of the chair, and are never seen again. The Greeks talk about putting incompetents “in the fridge”—giving them pretend jobs. In France it is the cupboard. Americans refer to “the dance of the lemons”—the practice of reassigning bad teachers to new schools rather than getting rid of them. They also refer to the “rubber room” where incompetent or criminal teachers bounce around, often for years, while administrators and unions haggle over what is to be done with them.

Fattening Leviathan

The unions' influence extends to the size and nature of the public sector. Private-sector unions have learned to exercise self-restraint when it comes to pushing for more manpower: they realise that more workers may reduce the wages of their members and that a higher wage bill may drive their employers out of business. But public-sector unions are relentless in demanding more resources and more personnel, which conveniently translate into more members and more dues.

Their most dramatic success has been in Britain. When Britain's union-backed New Labour government came to power in 1997, public spending accounted for almost 40% of GDP. When it left power in 2010 public spending was nearly 50% of GDP (partly, to be fair, as a result of recession), and 1m workers had been added to the public-sector payrolls. In California, as Mr DiSalvo points out, the prison guards' union has been one of the leading advocates of getting tough on crime. The result of this policy has been a dramatic increase in both the size of the state's prison-industrial complex (from 12 prisons in 1980 to 33 in 2000) and the pay of the people who run it (prison guards in 2006 made $70,000 a year in base salary and $100,000 with overtime). But public-sector unions can prosper simply by opposing rationalisation: Buffalo, in New York state, has as many public workers in 2006 as it did in 1950, despite the fact that the city has lost half its population.

Public-sector unions combine support for higher spending with vigorous opposition to more accountability. Almost everywhere they have demonised competition, transparency and flexible pay. Teachers' unions have often acted as the Praetorian Guard in this fight. In Poland they are up in arms against attempts to increase the number of hours a week (a mere 18) they have to spend teaching. In São Paulo state, in Brazil, teachers have organised huge marches against government attempts to link promotion to performance and to reduce the number of days they can take off without notice. In Greece they have fought four consecutive education ministers from different parties over performance reviews. In Britain they are trying to kill “free” schools, which can be set up outside local-authority control. In America they have fought relentlessly against charter schools (which escape union rules about pay and promotion) and scholarship schemes (which give choice to parents).

The teachers' unions have an impressive record of terminating reformers. When Marietta Giannakou, the education minister in the last New Democracy government in Greece, insisted on teacher accountability, she lost her seat at the next election. Michelle Rhee, the chancellor of the awful school system in Washington, DC, closed failing schools, fired more than 200 ineffective teachers and principals, and advocated merit pay. But the unions fought her every step of the way, using their muscle first to get rid of her patron, the city's mayor, and then to bring about her own resignation.

It is impossible to calculate the cost of the unions' inflexibility. But several recent studies provide some indications. Policy Exchange, a conservative think-tank, calculates that people in the British private sector work 23% more hours than their public-sector counterparts over their lifetimes, thanks to public-sector strikes, sick days and early retirement. Barry Bluestone, a left-wing economist, calculates that the price of America's public services increased by 41% in 2000-08, while that of private services rose by 27%. Eric Hanushek, an economist at Stanford University, argues that replacing the bottom 5-8% of American teachers with merely average performers could move the United States from near the bottom to near the top of the international maths and science rankings.

The rigidity of the public sector does not merely reduce the quality of services. It also discourages innovation. In the private sector innovative firms routinely experiment with new business models, measure the success of those models and then expand successful ones. But whenever public-sector managers have tried to do the same—by establishing magnet schools that focus on certain subjects, or charter schools with longer teaching days, for example—the unions have opposed them. In France they have blocked any attempt to introduce more flexibility into the country's highly centralised education system, or indeed to change it at all.

Only Germany provides a chink of light. There, although around 60% of public-sector workers are unionised, wage increases in the public sector have lagged behind those in the private sector. And though civil servants, who make up nearly half the public-sector workforce, enjoy both special pension schemes and job security, they are not allowed to strike. Indeed, the idea of going on strike for political reasons is unthinkable among all public-sector workers in Germany.

The battle ahead

Public-sector unions now face the biggest challenge in their history. Governments almost everywhere—particularly in the rich world—are being forced to cut back public spending. Many governments (for example in Ireland, Greece and Spain) are cutting public-sector pay. Others (for example in Japan and America) are freezing it. Greece is increasing the retirement age from 58 to 63 and making it possible to fire public servants. Britain is cutting government departments by as much as a quarter, and is reviewing pensions.

In the United States several rising Republican governors are keen to turn the short-term struggle over pay and benefits into a bigger battle about trade-union power. New Jersey's Chris Christie (see article) and Minnesota's Tim Pawlenty have both eagerly taken on the new “privileged class” of public-sector workers. Do the public exist to serve public-sector workers with their high pay and inflated benefits, they ask, or do public-sector workers exist to serve the public?

Even people on the left are beginning to echo these complaints. Andrew Cuomo, the incoming Democratic governor of New York, is rattling his sabre against public-sector unions despite the fact that they make up an important part of his base. Davis Guggenheim, an impeccably liberal film director whose credits include Al Gore's “An Inconvenient Truth”, subjected the teachers' unions to a merciless critique in “Waiting for Superman”, flagellating them for perpetuating a broken system and presenting Randi Weingarten, the head of the American Federation of Teachers, as “something of a foaming satanic beast”, as the Variety reviewer put it.

The unions have responded by proclaiming war on cost-cutting governments. They have already organised strikes and protests. Millions of French workers marched against Nicolas Sarkozy's modest plans to raise the retirement age by two years. Hundreds of thousands of people have taken to the streets in Ireland and Greece against austerity measures. London Underground workers have repeatedly paralysed transport in the city. But this is a mere prelude. Unions across Europe have promised strikes in 2011 on a scale not seen since the 1980s.

Public-sector unions will find it hard to win these battles. They have not been particularly successful in mobilising public anger, considering the scale of the cutbacks. Nor have they notched up any notable victories: the Greek and Irish governments have implemented their austerity packages and Nicolas Sarkozy has raised the retirement age. They are also discovering that many people in the private sector regard their public-sector colleagues as an overprivileged elite. Spanish civil servants were shocked at how little support they got when, last June, they protested against a 5% cut in pay. And a recent poll showed that 65% of people in stick-in-the-mud Greece want civil servants to lose their job security.

The pressure to rationalise the public sector is likely to continue in coming years. The debt level in OECD countries is expected to rise to 120% of GDP by 2014, thanks to a combination of ageing populations and inherited obligations, some of them driven by the public sector's insatiable appetite for pensions. Joshua Rauh, of the Kellogg School of Management at Northwestern University, reckons that seven American states will have exhausted their pension assets by 2020.

It would be a mistake to write off the public-sector unions. They are masters of diverting attention from strategic to tactical questions. Undoubtedly the unions will lose some of their privileges over the coming years; the scale of the debt crisis makes this inevitable. But will governments have the courage to tackle the root causes of the problem (such as pensions) rather than dealing with secondary problems (such as wages)? And will they dare to tackle questions of power rather than just pay and perks? If they are to claim victory in the coming fight, they need not just to restore the public finances to health. They also need to breathe the spirit of innovation into Leviathan.