Theresa May is under intense pressure to abandon cross-party Brexit talks, after a group of senior Conservative figures, including leadership contenders Boris Johnson and Dominic Raab, issued a strongly worded warning against any deal that involved a customs union.

May’s cabinet, and Jeremy Corbyn’s shadow cabinet, are both due to take stock of the talks on Tuesday, with neither side optimistic about the prospects for an agreement that could secure a majority in parliament.

Meanwhile, the prime minister’s Brexit negotiator, Olly Robbins, was said to be travelling to Brussels for talks with EU officials on whether the political declaration could be changed, in a bid to break the deadlock.

The BBC said Robbins will explore how quickly changes could be made to the declaration if the government and Labour could come to an agreement.

In a letter to the prime minister, former cabinet ministers including Gavin Williamson, who was sacked earlier this month over the Huawei leak, the former Brexit secretaries David Davis and Raab, and former foreign secretary Johnson, cautioned her against any deal involving Corbyn’s central demand of a customs union.

Q&A What is a customs union? Show A customs union means that countries agree to apply no or very low tariffs to goods sold between them, and to collectively apply the same tariffs to imported goods from the rest of the world. International trade deals are then negotiated by the bloc as a whole. For the EU, this means deals are negotiated by by Brussels, although individual member state governments agree the mandate and approve the final deal. The EU has trade deals covering 69 countries, including Canada and South Korea, which the UK has been attempting to roll over into post-Brexit bilateral agreements. Proponents of an independent UK trade policy outside the EU customs union say Britain must forge its own deals if it is to take advantage of the world’s fastest-growing economies. However they have never explained why Germany manages to export more than three times the value in goods to China than Britain does, while also being in the EU customs union. Jennifer Rankin

They told her such a move would be “bad policy and bad politics”, as it would contradict the 2017 Conservative general election manifesto and force the UK to open up its markets automatically to any countries with which the EU strikes trade deals in future.

“We believe that a customs union-based deal with Labour will very likely lose the support of Conservative MPs like us who backed the withdrawal agreement in March (in many cases very reluctantly), and you would be unlikely to gain as many Labour MPs to compensate,” said the letter, obtained by the Times.

“More fundamentally, you would have lost the loyal middle of the Conservative party, split our party and with likely nothing positive to show for it. No leader can bind his or her successor, so the deal would likely be at best temporary, at worst illusory.”

The fragility of May’s leadership, and the threat it presents to any deal being deliverable, has been discussed openly in the cross-party Brexit talks, which resumed on Monday.

Labour is demanding what it calls “entrenchment”, a legal underpinning for any promises made by the prime minister, to prevent a future leader unpicking them.

In signing the letter, Johnson and Raab underlined Labour’s fears, by making clear they would aim to sweep away any deal that involved a customs union.

The presence of 1922 Committee chair Sir Graham Brady’s signature on the letter is also significant, as he is charged with the job of conveying backbenchers’ views to the prime minister.

The committee’s executive is expected to return for a third time this week to the issue of whether Tory leadership rules should be changed to allow a fresh challenge to the prime minister.

May had been keen to table the withdrawal agreement bill (WAB) before the European elections on 23 May, but the mood of Conservative MPs has been hardening against it, even without any customs union concession.

A number of Eurosceptic Conservative MPs were already planning to switch back to rejecting May’s deal – adding to the 28 leavers and six remainers from the party who held out against it last time. Tory MPs said support for May’s deal was “peeling off” as the prime minister’s power was ebbing away and they wanted to show their members and constituents they were serious about a hard Brexit.

Eurosceptic Steve Baker indicated over the weekend that he would stand in the leadership contest if there was no other candidate who clearly opposed the withdrawal agreement, suggesting that Raab and Johnson could be outflanked on the right unless they take a position against it. If Baker does not stand, another of the 28 “Brexit ultras” is likely to put themselves forward.

Other Tory MPs suggested that May’s agreement was growing more unpopular daily as polls demonstrate the extent of support for Nigel Farage’s Brexit party, which is endorsing a no-deal position.

Such is the concern about support draining away from the deal, that ministers have discussed breaking up the legislation, into what one government source called “bite-sized WABs”, in a bid to ratify less controversial aspects.

The aim would be to demonstrate some progress in delivering Brexit, hopefully before polling day, though ministers are increasingly pessimistic about meeting that deadline.

Such a move would be likely to be regarded with exasperation by Britain’s EU negotiating partners, who signed off on the withdrawal agreement as a package.

Cabinet is expected to discuss whether to press ahead with any more Brexit talks with Labour or move to a series of parliamentary votes with a binding outcome – or “definitive votes”.

The cross-party negotiations broke up on Monday night with both sides once again signalling that no substantive progress had been made.

“Today’s meeting took stock across the range of issues discussed in talks over the last few weeks,” said a Downing Street spokesperson. “We continue to seek to agree a way forward in order to secure our orderly withdrawal from the EU.”

A Labour spokesperson said: “Talks continued tonight and the shadow cabinet and the trades unions will be updated.”

Like May, Corbyn is constrained by his backbenchers. He came under pressure from MPs at the weekly meeting of the parliamentary Labour party on Monday night over what several claimed was the party’s fuzzy stance on Brexit and lacklustre approach to the European election campaign.

Ilford North MP Wes Streeting said: “We’re trying to please everyone and instead are pleasing no one.”

Some in Labour are keen to make a deal but Keir Starmer, the shadow Brexit secretary, warned in the Guardian on Monday that about 150 Labour MPs would be unlikely to back it without a second referendum, which May has ruled out.

Even if no clear Tory-Labour deal is agreed, No 10 has been hoping to reach a pact with Labour that the opposition would allow the bill to be waved through on second reading, with MPs then able to find a consensus for an acceptable form of Brexit through amendments.

One option may be for May to publish the withdrawal agreement bill but not allow its second reading just yet. She would instead move on to the “definitive votes” in the Commons, whereby MPs would express a preference for a form of Brexit they could live with – potentially involving a customs union or a confirmatory vote on any deal.

However, some Brexit-supporting Conservative MPs scoffed at this prospect, saying it was inconceivable that May would remain in post long enough for this to happen.

One member of the 1922 Committee said opinion on the executive was still deeply split on the issue of May’s future but a win for Farage at the European elections was likely to be the pivotal moment at which opinion shifted in favour of allowing MPs a ballot on her leadership.