Either the poorest in society or the “hard-working people” courted by the Conservatives face being targeted under the party’s commitment to £12bn of welfare cuts, experts have said.

One way of achieving the £12bn goal could be by reducing the £38bn cost of out-of-work payments to working-age families, for example by cutting entitlements to a third of the recipients, according to John Hills, director of the centre for analysis of social exclusion at the London School of Economics.

“But that would mean hitting lone parents and disabled people and create pressure on food banks and hardship on a scale that would be hard to imagine,” Hills said. “Alternatively you could take it from hardworking families who rely on housing benefit and tax credits. That’s a lot of pain from a large number of people who have just voted for you.”

For David Cameron, benefit reform is both a boon and a burden. While the Tories have succeeded in rallying support against high taxes and welfare provision on the basis of balancing the nation’s books, the party knows it can be tarred as “nasty” if the deep cuts are seen as unfair.

The problem in the next parliament is that the Conservatives, governing alone, are committed to around a 10% cut to the £125bn spent on social security excluding the state pension and universal pensioner benefits. Of this amount only £2bn has been identified.

With a black hole in the spending plans the issue repeatedly flared up, and the prime minister has fallen back on suggesting that he does not want to cut benefits while leaving the impression that he might have to.

To justify the cuts, the Tories are likely to employ a narrative of skivers v strivers, suggesting a clear division between a large, permanently welfare-dependent group and the rest of the population who pay taxes to support it. The Tories know this is a fiction, but it is a politically useful one. Welfare is mainly about taking money from those of working age – when incomes are high on average – and giving cash and services to older people, and families with children.

A DWP paper setting out options was leaked to the BBC in March. The welfare secretary, Iain Duncan Smith, said none of the measures had been agreed and the media coverage was “ill-informed speculation”. Nevertheless, a similar internal report was produced before the 2010 election and dusted off to be presented to the new coalition government, which implemented many of the measures including the bedroom tax and household benefit cap.

So if the BBC’s document is any guide, George Osborne – reappointed as chancellor – could look to strip £1bn from carers’ allowances; means-test national insurance-backed unemployment benefits, saving another £1.3bn; and tax disability benefits to raise another £1.5bn. Then there’s limiting child benefit to two children – affecting a million families to save another £1bn.

The Institute For Fiscal Studies noted: “These may well not be the decisions that a future Conservative government would make. But it is likely they would have to make changes at least as radical as this to find £12bn a year.”

Not all these changes would require a new bill but if past form is anything to go by then the Tories would want to lay a trap for their opponents with new legislation so as to paint anyone who votes against it – such as the anti-austerity Scottish Nationalists – as pro-welfare parties prepared to spend lavishly on the idle poor.

A key question is how Labour reacts. Under Ed Miliband and Ed Balls, Labour attacked the Tories for lacking compassion but ending up offering their own welfare cuts. The only benefit change they would have reversed had they taken power was the bedroom tax.

However, the Labour hierarchy never had a distinctive view of welfare. One person who did was Frank Field, the Labour MP whose views on poor people have been sought out by Thatcher, Blair and Cameron. His plan to pay for services in a “contributory” way offered perhaps a popular alternative for social security.

Under Field’s plans, everyone would be compelled to make income-related contributions into a mutual fund to access, say, unemployment benefits – so the rich pay more. The government would pay for those too poor to do so. Although set up by the state, the corporations would be run by elected officials. Voting would give people more power over what their money is spent on.

There is something in the idea that raising contributions is not politically toxic in the way that lifting taxes are. Gordon Brown’s “popular tax rise” was the 1% increase in national insurance to pay for the NHS, which raised £8bn in 2003. National insurance contributions – which the government has to spend on specified benefits – raise about £100bn a year.

As risky is pooled in insurance, the rich would make higher payments for negligibly extra benefits, which fits into Labour’s view of fairness. Rather than fixing irresponsible capitalism, what the next Labour team must answer is the question of how to make socialism responsible.