The argument of this lecture speaks directly to the history of the Political Quarterly and the LSE. It is that the European Left is losing elections on an unprecedented scale because it has lost control of the political agenda to a newly flexible right; but it is also losing key arguments about how to nurture human values in today's connected and competitive global village because it has not responded to changes in economy and society; and that to turn things round it needs to address both its deficit in ideas and organisation.

Political Quarterly was founded in 1930 by people who believed that economic liberalism was not liberal without social justice. The founders of the LSE, Sidney and Beatrice Webb, believed passionately that public policy could shape society for the greater good. Their brand of collectivism - which was high handed and centralist as well as high minded and egalitarian - was intended as an alternative to the challenge of Marxism on the Left and laissez-faire individualism on the Right. Very Third Way.

In the inter war period, the reformist left seemed to be coming into its own. Leading the fight against the far right in Italy and Spain. Taking power in Britain, Germany and Sweden. David Marquand has described this period as an "Indian summer of gradualism...the last chance for men of compromise and reason to shape their own societies and the international community in the optimistic image of reformist social democracy."

"The optimistic image of reformist social democracy." That was Europe before the tragedies of the late 1930s and early 1940s. Indian summer indeed.

But it is a very resonant phrase. And it powered the great struggles and great victories of the post war period. It achieved nothing less than the creation of the most civilised, egalitarian societies in the world based on individual rights and communal services.

And the question I want to address today is whether that's it. Whether the full employment fair societies of the post war period are the best that politics can buy.

Because it is not glad, confident morning in European social democracy today. Quite the opposite.

After a decade of extraordinary, unprecedented success in the 1990s, under the banner that in Britain was called New Labour, reformist social democracy seems to have been put in check by so called compassionate conservatism; and the question is whether it is check mate.

The right is seeking to emulate the electoral strategies of the left in the 1990s; and the left in the last decade has not been able to decide whether to disown them or embrace them, when the key is in fact to build on them. So it is losing elections again on a grand scale.

Look at the facts.

The British General Election 2010. The second worst result since 1918.

Sweden, also 2010. The worst result since 1911.

Germany, 2009. The worst result since the founding of the Federal Republic, with a greater loss of support than any party in the history of the country.

France, 2007. The worst result since 1969.

Holland, 2009. A traumatic transition from junior coalition partner to opposition.

Italy. A yo-yo in and out of power, with personal and political divisions disabling opposition to Berlusconi.

These six countries, with good claim to represent the historic heartland of European social democracy, the homes of heroes from Bevan to Gramsci to Brandt to Mitterrand to Palme, the place where revisionism as a credo was created, are now run by the centre-right.

And you know the last time that happened? I asked the House of Commons library. The answer: not since the First World War has there been this kind of domination from the right. The whole era of democratic suffrage.

The motto of the LSE is to "know the causes of things". That's exactly the challenge.

Sure, you can avert your eyes. You can say Spain, Portugal and Greece all have governments of the centre-left. And they do.

You can say that Barack Obama and Manmohan Singh show what is possible. And they are struggling manfully to fight the obscurantism of the Right. You can say the leaders were less popular than their parties. And it is certainly true that Gordon Brown and Mona Sahlin were much less popular than their parties in the recent British and Swedish elections. But leaders reflect their parties.

You can even say that the Right are going to make a terrible mess of their time in Government, because they are nowhere near as nice as they pretend, and their programmes are confused at best and dangerous at worst. The Chirac/Juppe government, elected in 1995 to heal "social fracture" quite quickly exacerbated social tension and let the left back in 1997. But while that is true, it is not enough; just ask the social democratic parties in France, Sweden, Germany who have lost twice in the noughties.

My view is that after we have considered all the contingent factors, all the cultural differences, and after we have forsworn the option of accepting that we are wrong and there is nothing better in politics than the right can offer, there is a fundamental question to be answered.

Left parties are losing elections more comprehensively than ever before. They are losing from government and from opposition; they are losing in majoritarian systems and PR systems; just for good measure they are losing whatever position the party had on the Iraq war; and they are fragmenting at just the time the right is uniting.

I don't believe this to be some accident or cosmic joke being played by destiny. There are real reasons that need to be understood if we are to move forward on any other basis than waiting for the right to run out of steam. And they are only properly visible if you look across the six countries and join the dots.

There is only one place to start. Where have the voters gone?

My answer is that if you look across the six countries there are three groups of voters we on the centre left are losing. All three groups have a class base and a set of values that they feel have been violated by the centre-left.

Centre left parties are losing working class voters to the far right and far left. Just look at the second place for the anti Islamic Party for Freedom in Holland. There are two related reasons, in which interests and culture are interwoven. These voters find immigration to be a very big issue on which the centre-left is suspect at best and guilty at worst; and they find that their jobs are the first to go in the new economy. So while the Front National started in France in the 1980s as a problem for the Right, they and their sister parties around Europe are a bigger problem today for the left.

And in some PR systems, like Germany, there is a double squeeze with some of these voters going to the far left.

This splintering of the left, and the coalition politics it invites - "plural left" in France, Red-Green-Left in Sweden - is an electoral problem in itself, and has a ripple effect on a second group of voters. Centre left parties are losing middle income, swing voters, often young parents, in part because of coalitions with the left and Greens. Just look at Sweden. Only one in five Stockholm residents voted for the social democrats; the figure for those in work was nearly one in ten (13 per cent); only half of trade unionists across Sweden voted for the social democrats; and a third of those voters who turned their backs on the social democrats say did so because of their alliance with the Left Party.

The primary reason is tax and spending issues. These voters have a good lifestyle and don't want to lose it. They certainly don't want to trade part of it in for more generous welfare systems.

In Britain, median wages stagnated after the dot com crash, in other words well before the financial crisis. This is the squeezed middle whose position Ed Miliband has effectively highlighted.

Centre left parties are in addition losing a further group of voters - often middle class voters, but also young voters - who feel turned off the compromises of power, have no truck with the right, but want a different alternative to the established parties. The Green and Lib Dem votes in Britain are some indication of that. In Germany the Greens are doing very well in national polls, though the Social Democrats did well in the recent Hamburg state elections.

If this is the electoral arithmentic, the second question is 'why have they gone?'.

In the 1990s the optimistic image of reformist social democracy spoke to the times. 13 of 15 EU governments came from the centre left in 1999.

Across Europe, reformed centre left parties built a narrative of fair but flexible labour markets, social investment in education, renewal of welfare and strong internationalism.

The parties were not all the same. Britain's experience in the rebound from Thatcherism was different from Sweden. But even in France, where social democracy was such a weak brand that when I once asked Jacques Delors why he didn't run for the Presidency his answer was that there were only three social democrats in France...even there, Lionel Jospin made the centrepiece of his politics the commitment to a 'market economy but not a market society'. Social change - women's rights, gay rights - proceeded apace.

Social investment was increased.

Welfare to work programmes seemed to work, poverty was attacked.

And Europe as a political project provided a binding economic glue on the Continent.

Revisionist social democratic politics dominated the post cold war era. It dominated the centre ground; it led to significant social reform.

What has changed? We need to look at economics, politics and ideas.

After the fall of the Soviet Union and its associated red scares, and before the rise of China, India and Brazil had really taken off, growth in Europe seemed secure. Some governments even talked about the end of boom and bust. Tony Crosland's dream, that in a time of full employment we could focus on equal opportunity, seemed to be coming into focus.

But the policies and politics that worked in the NICE decade of the 1990s aren't up to the job in the GRIM decade that we are now living through.

Mervyn King coined the idea of the NICE decade. Non Inflationary Continuous Expansion.

Instead of a NICE decade we face a GRIM decade. Growth Restricted and Inflationary Misery.

It is true that out of the 1930s came the New Deal and the Keynesian welfare state. But Professor Andrew Gamble has explained that recessions rarely bring short term political benefit the centre left. Just look at the 1930s, the 1970s and the current period.

It is ironic but deeply indicative that it takes a man with the economic credibility of IMF Managing Director to give the French left its best chance of winning its first Presidential election in four.

Politics across Europe is not determined by economics, but it is shaped by it. And in the face of severe global competition, the crunch on growth and the distribution of its rewards has consequences. Politics has taken on a harsher hue - on welfare and wages, on tax and spending, on immigration - to the benefit of the right. And the increased budget deficits, the symptom of the expansionary budget antidote to slump, has provided a new and simple rationale for the centre right.

Yet it would be foolish not to recognise a second and decisive factor in the recent losses of the centre-left: the electoral detoxification of the Right. After its successive beatings from Clinton, Blair, Persson, Kok, Prodi, Schroeder, the Right re-grouped.

George Bush II showed how to win - well, kind of win - in 2000. He ran as a compassionate conservative. He ran against the Republican East Coast establishment, championed education and even progressive immigration reform.

In Europe, parties of the right realised that they had been pushed off the centre ground, and they responded.

Where once right of centre parties seemed anti deluvian on social issues, they embraced a new world of equal gay and women's rights.

Where they seemed in hoc to the rich, they upped the rhetoric against the unacceptable faces of capitalism. It wasn't President Hu of China who told the Davos meeting this year that "Globalisation...gave rise to a world in which everything was given to financial capital and almost nothing to labour...in which those who lived on unearned income left the workers far behind." It was President Sarkozy.

Where they seemed out of touch with the modern world, as in the UK, the right went green.

And where they seemed plain antipathetic to the national character, as in Sweden, they accommodated to the centre ground. Swedish Conservatives went from 15 per cent of the vote in 2002 to becoming a twice election-winning Alliance of the Centre-Right in 2006 and 2010.

In other words, they triangulated back against the reformed left. Their slogan is not Old Right or New Left but New Right.

And the left has been unsure of whether to take the right's shift as a compliment, or as proof that it went wrong in the 1990s.

This plays out in the battle of ideas. Since the 1920s, there have been three constants in every successful social democratic programme: greater protection from the dangers of life, more power over your own life, and stronger communities in which to live your life. All three promises have come under strain in the last decade under the pressure of economic and social change.

First, the argument about how to protect people from risks associated with a global economy. The reformist left argument of the 1990s was, in Lionel Jospin's phrase, to manage globalisation not fight it. Central to that was an active welfare state. The old welfare state offered a residual safety net; the new welfare state of education and training would offer a trampoline.

But the downward escalator that makes people fear for their children's economic future has been stronger than the measures to promote social mobility. In fact welfare is seen as not tough enough by those who see idleness in benefit recipients, and not empowering enough for those on the receiving end.

One consequence is that the fairness argument has been turned to the right. This has been exacerbated by the shift in the tax base highlighted by Peter Kellner.

Two thirds of the British electorate in the 1950s and 1960s paid little or no income tax. That is no longer the case. The employment structure has changed. And spending has massively increased, with important positive effects, on health, education and welfare. For these voters social democracy in the form of health, education and welfare, has gone, in his phrase, from being an "unambiguous blessing to a contingent one" - contingent on the quality of what is offered against a right determined to contest this terrain.

The second argument is about how to give people more control over their own lives.

Historically, it was the market which treated people as commodities to be bought and sold, and which left people feeling defenceless and stranded. The role of the state was to empower people, first of all through the vote, then through rights, then through services.

But the argument has now been turned. The very success of social democrats in arguing for an extended role for government means that the understanding people had of the market - that it was a "good servant but a bad master" - is now applied to government. That is the explanation I have for how a market failure like the banking crisis becomes a government failure of regulation.

The association of the left with the state has become a stick with which it is beaten; and the very expansion of the role of government to meet popular demand has made it more vulnerable to the charge that it is a powerful ogre not a flimsy line of defence.

The tragedy in a country like Britain is that it takes a Tory government to remind people of why it was worth having a Labour government. But the electoral consequence thus far in the 21st century is that the investment argument has been turned against the left.

There is, thirdly, the argument about how to foster a modern sense of belonging. This is not only about immigration, or only about poorer voters. But it is significantly about both.

Jonathan Rutherford has written about a story of "dispossession...in the shadow of the bright lights of consumer culture and the glamour of celebrity and money". The cosmopolitanism of diversity and individual rights is perceived as threatening and alien. He cites the English Defence League - a self styled street militia ready to fight the 'civilisational threat' of Islam - as a symptom of cultural dislocations and economic crises.

The right have few answers on immigration, as the current UK government are showing with promises that cannot be met except through perverse decisions on questions like visas for foreign students. But the left is torn between commitment to individual human rights for all people whatever their nationality and a recognition that communities depend on deep roots and long standing.

The consequence is that we are on the back foot when it comes to community. The Searchlight research published last week should be required reading for anyone who wants to understand politics in Britain; it explains the danger of assuming a natural centre-left majority when values issues cut across political divides in a fundamental way.

So if we know who we have lost, and have some idea of why we have lost them, what next?

One conclusion is that only by reversing out of the Third Way cul de sac can the centre left find avenues for advance. It is certainly true that the centre-left governments of the 1990s were good at helping the poorest benefit from economic expansion, not good enough at figuring out how to spur that expansion. They were good at preaching responsibility for those on welfare, not good enough at demanding responsibility from those at the top of society. They were good at the analysis of an enabling state, but not good enough at bringing it about; good at the rhetoric of public sector reform, not good enough at delineating how both planning and markets are necessary for an effective public sector. And they were good at building electoral machines, not good enough at building movements of social change.

But my strategic view is essentially the opposite. The revisionism that was entailed in the renewal of the left parties in the 1990s was essential for them to become viable. It is not the new doctrines of the 1990s that made these parties unviable; it is that these doctrines staved off unviability, for parties that had become practised at losing elections in the 1970s and 1980s. The good things about progressive politics in the 1990s - a radicalism when it came to doctrine, new thinking about national and international reform, a finely tuned eye and ear for social and technological change, decisive engagement with people's needs on difficult issues like crime and security, a readiness to pursue social justice in new ways, a strong sense of international responsibility, and a record that did leave the countries they governed fairer but also better prepared for the modern world - are the basis of winning again.

In other words, only a post New Labour brand of European social democracy, building on success, not a pre New Labour stance, can address the weaknesses that were left and exist today.

The routemap to victory is not straightforward. It requires reconnection with disenchanted electorates through new ideas; through new mechanisms for organisation; and through renewed political strategies appropriate to each country.

My starting point is to go back to a text written by RH Tawney after the 1931 defeat. He said Labour needed "a common view of the life proper to human beings". That is the place to make emotional contact with people, and the raw fears and ambitions that motivate them.

In other words start with an ethic not a policy. An ethic which informs the most basic questions that people want to see addressed - about work, family, opportunity, responsibility. And then apply that ethic to the great questions of the day.

How to build a moral economy. Our vision is not just about how much money is made; it is about how it is made. We are not apologists for globalisation. We are reformers. When left of centre parties are able to fight elections as private sector reformers, in the name of efficiency and not just fairness, they can win. When they do so, and make government an ally in wealth creation and a defence against corporate abuse of power, they turn the antipathy of the right to government on its head. For example, the privatised utilities in Britain, including rail, are a big part of the economy. We never satisfactorily addressed their functioning in government. We now have a responsibility to think about how they serve the British economy.

How to build a decent community. Our vision is not limited to state and market. When we fight elections as public sector innovators as well as private sector reformers, we live out our most basic insight - that we are socialists not statists. We do not create virtuous people by bureaucratic methods. We will not expose the flaws of the Big Society through bigger government, but through a better recipe for the Good Society.

How to make globalisation sustainable. The centre-left can't afford to look like suckers; but we Europeans have pioneered a different view about how to share sovereignty in the modern world than the Americans or the Chinese. I call it Responsible Sovereignty. Yes the nation state is the foundation of legitimacy and identity. But the assertion of national sovereignty is not enough in an interdependent world, where any problem of health, crime, economy, security has an international as well as a national dimension.

The challenge is to develop a distinctive centre left vision for European policy. The choice is not just for or against Europe; it must be to think through how social democrats bring our politics to the European level. The perspectives and budgets of the EU on internal as well as foreign policy owe far too much to the 1960s and 1980s. The right's recipe of fiscal retrenchment offers little. We must forge an opportunity to be internationalists of a hard headed and serious kind, or our policy solutions will have no traction at all.

I want to make a final point about how these ideas are developed. In the 1990s, the renewal came through think tanks. That is important. But it is not enough.

Eight decades ago Tawney identified the perils of government as a giant problem solver: "When it ought to have called people to a long and arduous struggle, it too often did the opposite. It courted them with the hope of cheaply won benefits...It demanded too little and offered too much."

This is fundamental because we express our political soul in the way we do politics as much as our policies and programmes; and processes that are top down produce solutions that are top down. We need a different mindset.

I am spending some of my time on the Movement for Change - a leadership academy for community organisers. It is a new way of rebuilding the labour movement. It is starting small: just a few staff, but with a big idea - to train 10 000 people before the next election in the skills necessary to use power locally. These are going to be the people who think through how to make welfare work; how to build private sector capacity; now to make the public sector a real engine of partnership. Above all they will help build confidence in communities to be players and not just spectators in the dramas of life.

Because if you think about it, this should be a time when the reformist left comes into its own. The realities of an interdependent world speak to the deepest traditions of progressive politics.

The rationale of the centre left is not just that we are forced to share; it is that by sharing we create and not divide.

This is a world of shared risk - from wage to avian flu to terrorism.

It is a world of shared identity - a growing global consciousness of what it means to be human. How else is one to explain how the self immolation of a fruit seller in Tunisia sparks a crisis of legitimacy around the Arab world.

And it is increasingly a world with increasing resources for shared action, involving government but not restricted to government. And that is what we were founded to create. Because "together we achieve more than we can alone".

Now we need to do it in new ways. And that should inspire us not depress us.

Because although the right have occupied the terrain of economic competence, it is far from clear that they have any answers to the fundamental question of how Europe is to pay its way in the world.

Though they have colonised the politics of community, they are fatally fractured between liberals and libertarians, hug a hoodie and hang em and flog em.

They have neutralised the left through compassionate conservatism, but they are fatally broken over the relationship between social concern and austerity. The Big Society is an admission of weakness not a sign of strength.

So there is a lot to fight for. And a lot worth fighting for. Because losing elections is not just one of those things. It is damaging for the people we represent, the countries we inhabit, and I would argue too for the world we share.

Above all, it is not inevitable. That is the real lesson of the last decade.