So, millions of British workers are anxious and frustrated. Is anybody surprised at the precariousness revealed by the latest Skills and Employment Survey, published on Monday? The national survey, carried out every six years, shows that more employees feel insecure than at any time in 20 years; that work is being intensified, with people being asked to do more and work longer; and that for the first time people working in the public sector feel more insecure than those in the private sector.

The reasons for this are clear. All governments since Margaret Thatcher's have promoted flexible labour markets as the right response to globalisation, without radically altering the social protection system in order to cushion workers against the inevitable insecurities that arise.

And both the Labour and Conservative parties today refuse to confront the dire consequences of pursuing flexibility while simultaneously shifting social security towards means-testing and workfare, with more behaviour-testing of those on the margins – the unemployed, disabled people, minorities and youth in general. All parties now are also united in demonising migrants, the most flexible workers of all.

As long as this consensus prevails, more and more people will join the ranks of the "precariat", and the insecurities confronting people will continue to multiply and intensify. Politicians seem desperate to appeal to their respective versions of the so called "middle class", unable to empathise with the precariat and eager to dream up fresh and tougher sanctions against society's wounded. Labour's conversion to a strange contributions idea would make matters even more regressive.

Why public sector workers are under pressure is also clear. Early in this government, the minister in charge of public services said that he was determined to instil more fear among its workers. Congratulations on a job well done. But politicians should reflect on the well-documented fact that fearful, insecure people lose their sense of tolerance and altruism. Anxieties also weaken immune systems. Do not demand social responsibility from chronically insecure people.

Britain is not alone, of course. In Spain the precariat is now huge, struggling in a situation where at any moment over 50% of its youth is unemployed, and a quarter of adults work in the shadow economy. In Portugal, Greece and elsewhere the pattern is similar.

We know from these countries that, while the precariat is not yet a class for itself – one that is organised to act in its own interests – part of it is reacting to growing insecurities, inequalities and status frustration by supporting populist movements. Austerity makes matters worse.

Multinational capital needs a precariat: people habituated to unstable labour and unstable living; overqualified for what they are expected to do; and reduced to being supplicants, without the rights that mainstream citizens take for granted, and not knowing what to expect in an atmosphere of uncertainty.

It has been clear since the early 1980s that flexibility would result in fragmented labour markets in which millions would be made insecure, relying on casual jobs and erratic wages, without non-wage benefits and access to secure state benefits. Thatcher began the process; Tony Blair and Gordon Brown intensified it; and Iain Duncan Smith, George Osborne and David Cameron have applied the finishing touch – with Liam Byrne and others speaking a similar language of sanctions, workfare and denial of access to social housing for the most insecure groups in society.

Means-testing, the hallmark of New Labour, deepened poverty traps, leaving the woeful tax credit system to prop up a dysfunctional system of declining wages. The inevitable behaviour tests and precarity traps have generated a bog of insecurities for anybody falling out of the salariat and core jobs. The social insecurity system instils fear in those clinging on to jobs.

Instead of confronting this perfect storm, successive governments have built a workfare society, eagerly punishing the losers and allowing the fear of becoming a vagrant to haunt millions at night. What sort of mentality do these politicians have as they continue to paint a warped picture of "hard-working families" and worthy taxpayers dragged down by lazy scroungers?

We need an opposition to develop an alternative perspective, one that is based on rescuing a sense of social solidarity, empathy and compassion. Employees tend to be less anxious, the survey found, in workplaces where employers encourage employee involvement. But without a voice in the labour process, without a secure wage, without security-providing non-wage benefits, the systemic problems of the precariat will only deepen. Indeed, most in it are forced by circumstances to do work well below their qualifications and aspirations. And they know that such jobs only reduce their future chances of employment that is close to their aspirations.

We need a new model of social protection. Let us accept that jobs are not the magic solution – and that in a globalised market job guarantees are a false promise. Let us accept flexible labour too. But in return let us have a society in which everybody has a right to basic security and a more equal access to other insurance-based schemes. A multi-tier social protection system must be based on a modest basic income, so as to enable the precariat to build lives involving a balance of different types of work, not just labour in jobs.

In thinking this way, let us remember that progress always comes from imagining the desirably impossible, and then making it impossible to imagine doing without it.