Theresa May must either resign as prime minister or her party should force her out, the former Conservative leader Iain Duncan Smith has said.

Describing May as a “caretaker prime minister” after the Tories suffered their worst local election result since 1995, the hard Brexit supporter urged her to quickly announce her departure.

“We have to make a change,” he said in an interview with radio station LBC. “The [1922] committee has to sit again now, urgently, and decide that either the prime minister sets the immediate date for departure or, I’m afraid, [we] must do it for her.

“This is the only way – we have in a sense a caretaker prime minister at the moment. I think, therefore, that making fundamental decisions about where we go with [Brexit] would be a big mistake.”

Earlier this year, the prime minister survived a no-confidence vote in parliament that was forced by Labour leader Jeremy Corbyn, after the government suffered the largest defeat for a governing party in modern history when its EU withdrawal deal was rejected by a majority of 230.

It followed a similarly unsuccessful attempt from within May’s own party to oust her in December as party leader, when 117 Tory MPs voted against her, prompting her to state she would step down before the next election.

May’s authority has continued to ebb since, and potential leadership contenders have been publicly jostling for position. However, it remains unclear whether her own MPs would vote to bring down her government and risk a general election.

Following on from this week’s local elections in which the Tories lost more than 1,300 seats and Labour 81, Duncan Smith said there was a complete lack of trust in the leadership of both the main parties, but particularly the Conservatives, and poured scorn on the idea that May could continue to lead the party following her second “terrible election result”.

“What we have now is a leadership that … is deeply weakened by this last result,” he said. “It wasn’t equivocal, it was an absolute, clear and devastating judgment.”

He then described May’s attempts to arrive at a soft Brexit deal with the Labour party as absurd, and expressed his fear that Tory divisions could be perpetuated if the two parties agree on a deal leading to the UK’s continued membership of a customs union.

“The message was loud and clear that, since 29 March, people have decided they are absolutely furious with the political class,” the former work and pensions secretary said.

The Scottish Conservative leader Ruth Davidson said today that she believes Labour and the Tories are close to striking a compromise deal over Brexit. She also rejected calls for May to set a date for her resignation, saying the prime minister should be given space and time to negotiate.