Universal Credit (UC) is fast becoming an exemplar in policy failure. The programme is more than half a decade late, eye-wateringly over budget and peppered with unforeseen design flaws, which have incurred criticism from whistleblowers and acute hardship for those it is supposed to support. But the damage caused by Universal Credit does not hide the fact that the system it is replacing was also far from ideal. What answer do progressives have to offer on the future of social security?

On Universal Credit, the views of the National Audit Office are damning. Having initially been due for completion in October 2017, the Department for Work and Pensions now don’t expect Universal Credit to be fully rolled out until 2023. The ten per cent of people enrolled onto Universal Credit so far are among the most straightforward cases to manage. Yet running costs per claimant are already five per cent over budget, and four times higher than targeted by 2023.

Meanwhile a quarter of payments have been four weeks late on average, with one in five people affected by a late claim failing to receive anything for almost five months. This has coincided with foodbank use rising almost three times faster in areas where Universal Credit has been rolled out, compared to the average elsewhere.

With respect to its own stated objectives – increased simplification, reduced costs and strong work incentives – Universal Credit is already failing decisively on each of its own terms. Given government rhetoric, it is perhaps the third of these failings that is most surprising. But following cuts made to Universal Credit by George Osborne in 2015, the main family breadwinner on a minimum wage salary could keep just 25p in every extra £1 of earnings.

Nonetheless, assessing Universal Credit on its own terms is not enough. In a landscape where political contours and realities are fast changing, a deeper and wider conversation is needed about the very goals of 21st century social security themselves.

Certain principles, which feature more prominently in the social security systems of other countries, as well as in the UK’s own past, are conspicuous by their absence in the present reforms.

Missing, for example, is a convincing commitment to poverty alleviation or an adequate safety net. Replacing legacy benefits with Universal Credit is currently expected to push the rate of absolute child poverty up by three percentage points, compared to a world where the reforms are no longer implemented. The political desire to contain the size of the state has seemingly deprioritised the need to stop child poverty from rising, let alone reduce it.

Conditionality in Universal Credit also represents an unsatisfactory fudge between opposing principles of universalism and reciprocity. Today, most people are entitled to welfare support irrespective of their national insurance history (subject to means testing). But at the same time they have to demonstrably seek and take on future work or else face financial sanctions. The result is a confused and punitive narrative of welfare entitlement that does little to endear popular support.

These features also prevent people from entering the labour market on their own terms, hampering job quality and future work progression. People are forced to take the first lousy job that comes along, and productivity and pay – already desperately weak in the UK – suffer as a consequence.

Also missing is a desire to reflect the ever-changing family structures of the 21st century. Universal Credit assumes and rewards the traditional model of a single earner household. Work allowances – the amount someone can earn before benefits start to be taken away – in particular penalise the second highest earner in a household, which in turn disproportionately affects women. Not only does this extenuate existing labour market inequalities, but it also means Universal Credit is likely to prove brittle as family patterns continue to change and evolve over the coming decades.

But while Universal Credit falls short against many broad tests of progressive principle, the political “left” in the UK are also some way off from constructively engaging with the policy debate themselves.

On the one hand, there are those fighting in the weeds to affect the minutiae of Universal Credit, but at the cost of providing little opposition to the government’s broad agenda as a whole. On the other hand, there are many calling for a complete and total change in direction on social security – such as through the introduction of a “universal basic income” – but without offering the substance on how to get there, given the messy and imperfect system they would inevitably have to inherit first. Somehow, a radical but deliverable vision for progressive social security is largely falling between the gaps.

Filling these gaps requires three things. First, an open conversation about which core principles should underpin progressive social security – poverty alleviation, financial security, universalism, reciprocity, prevention over symptom control and rebalancing power towards those that have least of it, might all be considered. Where there are inevitable tensions between principles, these also need to be acknowledged and met head on.

Second, a recognition that social security needs to work in complement with public services. Recent cuts to local and central government provision have made people’s lives unquestionably harder, and put added stress on the social security system as well. Public investment in things like social care and homelessness services need to be seen as part of the same system as the provision of welfare.

Third, is the hardest bit. We need creative, radical but legislation-ready proposals for social security that make meaningful progress towards delivering the core principles, while recognising they need to start from the world as it is today.

There are a number of radical but realistic proposals out there, which, with further research and development, could fit the bill. For example, the personal allowance of income tax could be replaced with an equivalent cash payment for all, providing a non-conditional financial buffer without incurring significant additional cost to government. Going in a different direction, the state could also revolutionise national insurance to give people access to a personalised account, which is topped up by the state particularly for those on low or zero earnings and allowing people to smooth income more effectively while out of work.

The sense of direction needs to be clear and ambitious, but equally no one policy should carry the burden of achieving everything at once. And if the left want to provide a compelling offer for government, this work needs to start sooner rather than later.

Alfie Stirling is head of economics at the New Economics Foundation, he tweets from @alfie_stirling and Sarah Bedford is head of social policy at the New Economics Foundation, she tweets from @SarahBedford5.