Ministers have accused opposition MPs of scaremongering over universal credit after being forced to delay the full rollout of their flagship benefit system for a further nine months at a cost of £500m.

The Department for Work and Pensions admitted that the delay until 2024 – which means universal credit will be at least seven years behind its original completion date – was caused in part by claimants being scared to sign up for the new benefit.

Universal credit has been linked to increased rent arrears, debt, stress, and food bank use among its low-income claimants, who are forced to wait five weeks for a first payment after moving on to the benefit, and in many cases find themselves worse off and facing volatile monthly payments.

Formally announcing the latest delay in parliament on Tuesday – the previous delay came in October 2018 – the work and pensions minister Will Quince said fewer claimants than expected were moving on to universal credit, which he credited to more people being in work.

But in a BBC documentary to be broadcast on Tuesday evening, parts of which have been previewed by BBC News, it emerges that the director general of universal credit, Neil Couling, told fellow senior officials in the summer of 2019 that claimants were reluctant to move on to the new system.

He said: “We’ve got a lot of anecdotal evidence of people being scared to come to universal credit. It’s a potentially serious issue for us, in terms of completing the project by December 2023, but I’m urging people not to panic.”

A few weeks later, in September 2019, Couling decided to delay full rollout, saying it would be safer for claimants to do so even if it might prove controversial. “Three, six or nine months, it doesn’t matter – the headline will be: ‘Delay, disaster,’” he said, adding: “I’ll take the beating [for the delay decision].’”

The Labour leadership contender Lisa Nandy announced in a speech on Tuesday that she would like to see universal credit abolished.

“For too long the benefits system has been designed from the top down and the Tories have shown little regard for the brutal reality facing those who need it. Too many people end up destitute and in despair at the hands of a system that is supposed to help them. Universal credit should be scrapped and rebuilt with the people it should be supporting,” Nandy said in a speech to party activists, according to pre-released extracts.

Citing her experience in her Wigan constituency, where the new benefit has been rolled out, she said she would hold participative workshops to “bring people together and develop a new system – one that puts people back at the centre”.

Quince was asked in the Commons on Tuesday why the decision to delay the rollout had been kept under wraps for so long. He said senior officials discussed issuing advice on the delay to ministers in late 2019 and final discussions with ministers took place this year.

The Scottish Nationalist party MP Neil Gray, who lodged an urgent question on the issue, said: “Ministers say that this delay is caused in part because people are scared to go on to universal credit. They say it like they are surprised. The great irony in all of this is if this Tory government would listen and do what expert charities and those in receipt of universal credit say, then these delays would not be needed.”

Quick Guide What is universal credit and what are the problems? Show What is universal credit? Universal credit (UC) is the supposed flagship reform of the benefits system, rolling together six benefits into one, online-only system. The theoretical aim, for which there was general support across the political spectrum, was to simplify the system and increase the incentives for people to move off benefits into work. About 2 million people are currently in receipt of UC. More than 6 million will be on the benefit by the time it is fully rolled out. How long has it been around? The project was legislated for in 2011 under the auspices of its most vocal champion, Conservative MP Iain Duncan Smith. The plan was to roll it out by 2017. However, a series of management failures, expensive IT blunders and design faults mean it is now seven years behind schedule, and rollout will not be complete until 2024. The government admitted that the delay was caused in part by claimants being too scared to sign up to the new benefit. What is the biggest problem? The original design set out a minimum 42-day wait for a first payment to claimants when they moved to UC (in practice this is often up to 60 days). After sustained pressure, the government announced in the autumn 2017 budget that the wait would be reduced to 35 days from February 2018. This will partially mitigate the impact on many claimants of having no income for six weeks. The wait has led to rent arrears and evictions, hunger (food banks in UC areas report notable increases in referrals), use of expensive credit and mental distress. Ministers have expanded the availability of hardship loans (now repayable over a year) to help new claimants while they wait for payment. Housing benefit will now continue for an extra two weeks after the start of a UC claim. However, critics say the five-week wait is still too long and want it reduced to two or three weeks. Are there other problems? Plenty. Multibillion-pound cuts to work allowances imposed by the former chancellor George Osborne mean UC is far less generous than originally envisaged. According to the Resolution Foundation thinktank, about 2.5m low-income working households will be more than £1,000 a year worse off when they move to UC, reducing work incentives.

Landlords are worried that the level of rent arrears accrued by tenants on UC could lead to a rise in evictions. It's also not very user-friendly: claimants complain the system is complex, unreliable and difficult to manage, particularly if you have no internet access.

And there is concern that UC cannot deliver key promises: a critical study found it does not deliver savings, cannot prove it gets more people into work, and has plunged vulnerable claimants into hardship.

Margaret Greenwood, the shadow work and pensions secretary, said the delay was “hugely embarrassing” for ministers. “The government has been forced to delay universal credit yet again because people do not have confidence in the system,” she said.

Quince said the problems facing universal credit would be lessened “if the party opposite desisted with their scaremongering”.

About 2 million people are in receipt of universal credit, which bundles six working-age benefits into a single monthly payment. More than 6 million will be on the benefit by the time it is fully rolled out by the new completion date of September 2024.

Although ministers have promised income protection for claimants who find themselves worse off after moving to universal credit, this applies only to those who move under the managed transfer scheme starting later this year. The thousands who switch to universal credit from legacy benefits because of a change of personal circumstances – moving house, for example – do not qualify for protection.

Concerns about plunging public confidence in universal credit led the DWP to launch a £225,000 newspaper advertising campaign to “detoxify” the benefit last May. However, it was banned by the Advertising Standards Authority for being misleading.

A recent study of the impact of universal credit by the Resolution Foundation thinktank found that 46% of claimants nationally would be worse off on universal credit and 39% better off, with families in the most deprived areas more likely to lose out financially.