The Brexit negotiations are on a knife-edge as Theresa May’s domestic vulnerability over the Irish border threatens to kill off hopes of an October deal, with the Brexit secretary, Dominic Raab, forced to make a dash to Brussels to seek more time from the EU’s chief negotiator, Michel Barnier.

Days before the crunch leaders’ summit at which the EU has demanded “maximum progress” be made to allow the talks to develop, Raab made the unexpected flying visit.

He is understood to have told Barnier during a meeting that lasted just over an hour that the volatility of politics at home meant that a plan to strike an agreement on Monday on the backstop solution for avoiding a hard border on the island of Ireland was not possible.

Ambassadors for the EU27 were subsequently told on Sunday night by Sabine Weyand, the bloc’s deputy chief negotiator, that the “Brits need more time”. “The problem is at the British end,” a diplomat said.

“It was a sobering meeting because everyone had been hoping something would come out of the tunnel,” said a second diplomat. “But it was going to be a gamble to get a deal in October.”

The British government is facing huge domestic opposition to its proposal for a UK-wide, but temporary, customs union with the EU within the backstop, in which Northern Ireland would stay within the single market.

All negotiations between the EU and the UK have now been put on hold. The prime minister has been invited to address the EU27 heads of state and government before a dinner on Wednesday night but she is yet to accept.

Writing in the Daily Telegraph, the former foreign secretary Boris Johnson said the backstop idea as a whole should be jettisoned. “In presuming to change the constitutional arrangements of the United Kingdom, the EU is treating us with naked contempt,” he said. “Like some chess player triumphantly forking our king and our queen, the EU commission is offering the UK government what appears to be a binary choice. It is a choice between the break-up of this country, or the subjugation of this country, between separation or submission.”

Ireland’s ambassador to the UK, Adrian O’Neill, said events in Brussels were a “setback” and could increase the prospect of a no-deal Brexit. He told BBC Radio 4’s Westminster Hour: “Time is running out, there is no doubt about that. In all member states, preparation for all eventualities are ramping up quite significantly.”

A joint statement by the Brexit department and No 10 said a number of “unresolved issues” remained following Sunday’s talks between Raab and Barnier but the UK was “still committed to making progress” at Wednesday’s EU summit.

Raab has still not received a guarantee to ensure a proposed extension of Britain’s EU customs union membership – part of the backstop agreement – will be temporary.

A meeting of the most senior aides to the EU27, due to take place on Monday to sign off on an agreement on the backstop solution, has been cancelled.

Internal government emails leaked to the Observer revealed on Sunday that the Democratic Unionist party leader, Arlene Foster, had privately let it be known that she was readying herself for a no-deal scenario following a “hostile and difficult” meeting with Barnier last week.

The former Brexit secretary David Davis appealed on Sunday to the cabinet to “exert its collective authority” to kill off May’s plans, with as many as nine ministers said to be considering their positions.

Brexiter cabinet members have called a meeting to be chaired by the Commons leader, Andrea Leadsom, to discuss their next move on Monday. Previous attendees of such meetings have included the international trade secretary, Liam Fox, the international development secretary, Penny Mordaunt, the work and pensions secretary, Esther McVey, and the transport secretary, Chris Grayling.

Barnier tweeted on Sunday that “despite intense efforts, some key issues are still open, including the backstop for IE/NI to avoid a hard border”.

We met today @DominicRaab and UK negotiating team. Despite intense efforts, some key issues are still open, including the backstop for IE/NI to avoid a hard border. I will debrief the EU27 and @Europarl_EN on the #Brexit negotiations. — Michel Barnier (@MichelBarnier) October 14, 2018

The scene is set for May to make an appeal for such a deal over the dinner on Wednesday night, in a moment of drama that might prompt a helpful intervention from the assembled leaders. “This does feel like a bit of theatre is being injected to help her along,” one diplomat said.

EU officials warned, however, that the process could be derailed, with a heightened risk of a humiliation similar to that endured by the prime minister at the recent Salzburg summit.

In the event of the negotiating teams finding a compromise position on avoiding a hard border on the island of Ireland, a Brexit summit to finalise the terms of the political declaration on a future trade relationship has been pencilled in for the weekend of 17-18 November.

The French president, Emmanuel Macron, has said he would not attend a special Brexit summit in November unless there was progress on the Irish border in October.

The Guardian revealed on Sunday that the special meeting of heads of state and government, at which the EU had hoped to sign off on the Brexit negotiations next month, could instead be turned into an emergency summit to discuss the bloc’s response to a cliff-edge Brexit should progress on the Irish border prove elusive.

The European commission’s secretary general, Martin Selmayr, told member states last month that their governments would need to decide how far it would be in the EU’s interests to mitigate some of the impacts of a scenario in which the UK leaves the bloc without a deal.

A senior EU diplomat said leaders at the November summit, should it be focused on no-deal preparations, would also want to coordinate their responses in areas where national governments have competence, such as contingency measures to avoid long queues of lorries waiting at customs or in aviation and haulage.

One senior EU diplomat said: “Preparations on contingency are really advancing in almost all member states. We’re in close contact with our neighbours and exchanging all these issues. The commission has beefed up its team working on contingency.

“This is a parallel track. We’re going to do this anyhow whatever the outcome because even if there’s a positive outcome [this week] we’ll still need to continue preparedness and contingency because we can never exclude the possibility that negotiations will break down at a later stage.

“Unless we have a final deal agreed upon by the House of Commons and the British parliamentary system, we’ll carry on with our preparedness and contingency. That’s normal work.”

The UK and EU agreed in December, and again in March, that the withdrawal agreement would contain an Irish protocol in which a backstop solution for avoiding a hard border under any circumstances would be spelled out.

The EU has accepted in principle the British government’s argument that the backstop can include a UK-wide customs union. One diplomat said: “The wedding knot has been tied.”

Some diplomats in Brussels concluded that the UK could have publicly agreed the backstop this weekend, but was reluctant to make concessions without a detailed offer on the future relationship. “It was too difficult for the UK to give the impression that the withdrawal agreement was settled and move on to discuss the political declaration,” said a diplomat. “I am not too surprised that the UK could not accept a separate package.”

The British government has always objected to the EU’s sequencing of the talks, which requires progress on the divorce treaty before moving on to the future relationship.

EU diplomats expressed their disappointment, but were keen to show they were taking the missed deadline in their stride. “Negotiations have ups, they have downs, we wait for the regroup,” said one senior source. “It was good to put the pressure on now, so we aren’t in this situation in November.”