Theresa May’s prospects of getting her Brexit deal through parliament this week dramatically receded on Sunday night after a high-stakes summit with Boris Johnson and other leading hard-Brexiters at her country retreat broke up without agreement.

Tory rebels present said that the prime minister repeated “all the same lines” about her deal and that nothing new emerged during the three-hour meeting, at which Jacob Rees-Mogg, Iain Duncan Smith and Dominic Raab were also present.

One source said May was told by some of those present, including Rees-Mogg, that to get her Brexit deal through she needed to spell out when she was quitting No 10 so that another prime minister could lead the next phase of EU trade negotiations. But the prime minister did not respond to the suggestion.

The talks took place amid reports of an imminent coup to remove the prime minister – claims which were forcefully denied by Michael Gove, David Lidington and Philip Hammond.

But before a critical cabinet meeting on Monday morning, May remained in a perilous position, with no breakthrough and Downing Street only able to tell reporters that she had discussed “whether there is sufficient support” to hold a meaningful vote this week. A front-page editorial in Monday’s Sun urges May to quit, with the headline “Time’s Up, Theresa”, saying she should announce that she will stand down as soon as her Brexit deal is approved and the UK leaves the EU.

The Guardian view on Conservative crisis: made by Brexit | Editorial Read more

MPs are due to vote on Monday night on whether to take control of the parliamentary agenda and hold a series of indicative votes on alternative options, including a customs union and a second referendum.

That could leave the prime minister at risk of losing control of the Brexit process, although there was speculation on Sunday night that she may announce her own version of the multiple choice plan on Monday morning.

Play Video 6:17 'People’s vote' march: up close with anti-Brexit protesters at the 'biggest ever demo' – video

As the prime minister’s allies sought to play down speculation over the reported coup, senior members of the European Research Group and former ministers arrived at Chequers, where Duncan Smith appeared in an open-topped sports car – while Rees-Mogg brought along one of his sons.

Others in attendance included David Davis, a former Brexit secretary, Steve Baker, a leading hard Brexiter, and May ally Damian Green.

The prime minister, the only woman present, also invited her effective deputy, Lidington, and the environment secretary, Gove, plus Julian Smith, the chief whip, and Brandon Lewis, the Conservative party chairman.

Cabinet colleagues at the meeting, including Gove, had hoped it would be possible to persuade Johnson and Raab – both rival Tory leadership contenders who are holding out against the deal – to end their standoff by dropping their opposition together.

How could Theresa May be deposed? Read more

However, any compromise from Johnson appeared unlikely as he used his latest Telegraph column to attack the government, saying it had “bottled” Brexit and the UK should leave the EU immediately.

One argument advanced at the meeting by the government’s side was that if MPs were permitted to hold indicative votes on their preferred Brexit option, “the most likely scenario would be that they would go for a softer Brexit” – a threat that it was hoped might bring the rebels into line.

They also believed that Duncan Smith and Rees-Mogg were looking for reasons to sign up to a Brexit deal if May could offer them a plausible justification for ending months of opposition to her deal.

After the meeting, Downing Street released a short statement confirming the attendees and adding: “The PM and a number of government ministers met today at Chequers for lengthy talks with senior colleagues about delivering Brexit. The meeting discussed a range of issues, including whether there is sufficient support in the Commons to bring back a meaningful vote this week.”

May had been expected to try to bring her deal back to the Commons on Tuesday, but that now looks increasingly unlikely given that Johnson and other high-profile holdouts seem unwilling to change their minds.

Earlier on Sunday, May appeared to have seen off talk of a plot to replace her with Lidington or Gove as a caretaker prime minister. The idea of appointing Lidington was said to be supported by soft-Brexit ministers such as Greg Clark, the business secretary, and Amber Rudd, the work and pensions secretary, while Gove was said to have the backing of others, including Liz Truss, the chief secretary to the Treasury.

May’s time is up. She must make way for a caretaker prime minister | Matthew d’Ancona Read more

Lidington and Gove were quick to deny the speculation. “I don’t think that I’ve any wish to take over from the PM, who I think is doing a fantastic job,” Lidington said. “I tell you this: one thing that working closely with the prime minister does is cure you completely of any lingering shred of ambition to want to do that task.” Gove said he “absolutely” supported the prime minister.

Play Video 0:52 David Lidington: 'I don't think that I've any wish to take over from the PM' – video

Although few took the idea of an immediate plot seriously, May’s failure to make progress during the weekend will increase the fragility of her position with senior colleagues.

Cabinet is expected to discuss whether it will be necessary to hold indicative votes as an alternative after signs of sharp disagreements between leading ministers on the issue. Earlier on Sunday, Hammond admitted that May may not be able to get her deal through the Commons and that Tory MPs he had spoken to were very frustrated and desperate to find a way forward.

The chancellor suggested that indicative votes could be the answer. He said: “I’m realistic that we may not be able to get a majority for the prime minister’s deal and if that is the case then parliament will have to decide not just what it’s against, but what it is for.”

He said he could not support no deal or no Brexit, but that a second referendum was an idea that deserved to be considered.

But there was sharp opposition from Steve Barclay, the Brexit secretary, who said that if MPs voted for something that was not in the Conservative manifesto, which ruled out staying in the customs union and single market, that could provoke a constitutional crisis – or a general election.

May is expected to meet Jeremy Corbyn for a one-on-one meeting on Monday morning to discuss how to resolve the Brexit crisis. Labour has accused the government of indicating that it would try to ignore the will of parliament.

Keir Starmer, the shadow Brexit secretary, said: “Having lost control of events, the Brexit secretary indicates the government would frustrate any attempt by parliament to break the deadlock. We are only in this crisis because ministers have pushed parliament away at every turn.”