David Cameron and George Osborne were struggling to maintain a semblance of unity in the Tory party as allies of Iain Duncan Smith said he had quit the cabinet because he regarded their cuts to welfare as “morally indefensible”.

The extraordinary accusation from the Duncan Smith camp came the day after he sensationally quit as work and pensions secretary in a bitter row over £4bn of cuts to benefits for the disabled, which were announced in Osborne’s budget last Wednesday.

On Saturday, as the Conservative party descended into a savage war of words over the reasons for the resignation, friends of the prime minister and chancellor tried to undermine efforts by Duncan Smith to occupy the moral high ground, insisting he had personally agreed to the cuts last week and supported them at cabinet on the morning of the budget.

They also suggested that Duncan Smith – a supporter of Brexit – had been looking for several weeks for an opportunity to resign, and claimed that he wanted to find a moment when he could inflict maximum damage on the campaign led by Cameron and Osborne to keep Britain in the European Union.

The prime minister has admitted for the first time his “fear” Britain could sleepwalk out of the EU. He told the Independent on Sunday that the result of the 23 June referendum was now on a knife-edge and revealed his biggest concern was in persuading enough people to turn out to deny victory to the Brexit campaign.

A spokeswoman for Duncan Smith said it was “total rubbish” that he had resigned in order to advance the cause of Brexit, or that he had planned to do so for some time: “Downing Street are briefing this but it is simply not true.”

Duncan Smith’s allies said he had reluctantly gone along with the disability cuts in advance of the budget but had decided to quit when he felt he was being made to “carry the can” for the policy. They also said that he had decided to go when he was told by Cameron on Friday that, although the policy announced in the budget would not be implemented in that form, he would still have to find an equivalent amount from the disability budget. “This is the dishonesty of it,” said an IDS aide. “It is not a U-turn at all.”

Writing in the Observer, Bernard Jenkin, a Tory MP and chair of the Commons public administration select committee, says that Duncan Smith was not prepared to tolerate another raid on the disability budget.

Referring to the prime minister’s letter to Duncan Smith, in which Cameron said he was “puzzled” by the resignation, Jenkin writes: “What that letter does not make clear is that the £4bn savings in the budget from welfare still stands and, once again, Iain was being told to find similar cuts from other benefits for working-age people – including for the disabled – again undermining the positive incentives that make it worthwhile for them to take work. That is what he finds morally indefensible.”

It was unclear whether Duncan Smith would lay out his position further in a resignation statement in the House of Commons. He is due to appear on Sunday morning on the BBC’s Andrew Marr programme.

The resignation and row over disability cuts is widely seen among Conservatives as a severe blow to Osborne’s chances of leading the party when Cameron steps down, and as a potential boost to the chances of his great rival for the succession, Boris Johnson.

Facebook Twitter Pinterest Stephen Crabb is being urged to appear before MPs on Monday to say the cuts have been dropped. Photograph: Niklas Halle'N/AFP/Getty Images

Cameron replaced Duncan Smith at the Department of Work and Pensions with Stephen Crabb, the Welsh secretary, who was brought up by a single mother on benefits. Alun Cairn was promoted to the role of Welsh secretary.

In his resignation letter to the prime minister, Duncan Smith wrote: “I have for some time and rather reluctantly come to believe that the latest changes to benefits to the disabled and the context in which they’ve been made are a compromise too far.

“While they are defensible in narrow terms, given the continuing deficit, they are not defensible in the way they were placed within a budget that benefits higher-earning taxpayers. They should have instead been part of a wider process to engage others in finding the best way to better focus resources on those most in need.

“I am unable to watch passively while certain policies are enacted in order to meet the fiscal self-imposed restraints that I believe are more and more perceived as distinctly political rather than in the national economic interest.”

Duncan Smith has been at loggerheads with Cameron over whether Britain should stay in the EU, joining other cabinet ministers in calling for Brexit.

Labour urged Crabb to appear before MPs on Monday to announce formally that the cuts to disability benefits had been dropped. Owen Smith, the shadow work and pensions secretary, said: “His very first act as secretary of state must be to come to parliament on Monday to announce the full reversal of cruel Tory cuts that will see 370,000 disabled people lose £3,500 a year.”

He also urged Crabb to “stand up to a Treasury that is intent on cutting support for those most in need to pay for tax breaks for those who least need them”.