Downing Street has flatly ruled out customs union membership, before the cross-party Brexit talks Theresa May promised on Tuesday night have even begun.

The prime minister responded to Tuesday’s historic defeat in the meaningful vote by pledging to speak to “senior parliamentarians” to identify a deal that could secure a majority.

But the Labour frontbench position is for a permanent customs union, as is that of Conservative backers of a Norway-style Brexit deal, making it unlikely talks with either group would get off the ground if May stands by that red line.

Brexit: MPs debate no-confidence motion after May's deal defeat – Politics live Read more

Speaking to journalists after prime minister’s questions on Wednesday, a spokesman for May said: “The principles that govern us as we go into these talks is that we want to be able to do our own trade deals, and that is incompatible with a customs union.”

May had said at PMQs that while she was seeking to learn “what could command the support of this house and deliver on the referendum”, any proposal must involve “opening up new opportunities to trade with the rest of the world”.

Committing to a customs union would trigger a furious backlash among Brexit-backing Tory MPs – and Downing Street needs their support, and that of the DUP’s 10 representatives, to survive Wednesday night’s no-confidence vote.

“We have to win a vote of no-confidence this evening,” the spokesman said. “If we do that, we want to see talks with senior parliamentarians and voices in the House of Commons start quickly, and we’ve set out the approach that we’re going to take to these talks. That’s where we are.”

Earlier, the justice secretary, David Gauke – one of those cabinet ministers who pressed for May to reach out to other parties after Tuesday’s vote – had appeared to suggest the government is open to compromise.

“I don’t think we can today be boxing ourselves in. What we need to be doing is engaging across parliament, seeing what ideas emerge, where the support is for those particular ideas and at that point we need to make an assessment is this something negotiable with the European Union and something with majority support in the House of Commons?” he said. “Today is about making an assessment about where the numbers are.

“I think the right answer would be to leave the customs union but given where we are, we have to be open to proposals that are put forward and make an assessment on the way forward. If we are going to engage, we have to engage.”

Gauke did not rule out conversations with the Labour leader but suggested they would be futile. “I think there may be others who might be easier to work with,” he said.

Speaking in the Commons, May said the exercise was about “wanting to understand the views of parliamentarians so that we can identify what could command the support of this House”. However, she said engagement must align with what she defined as respecting the vote to leave, citing “ending free movement, a fairer deal for farmers and fishermen, opening up new opportunities to trade with the rest of the world and keeping good ties with our neighbours in Europe”.

Ken Clarke, the veteran pro-Europe Tory MP, urged May to consider that a customs union could command support. “I have had to accept the majority in the house is committed to the UK leaving the European Union; she must also accept that she must now modify her red lines that she set for herself at Lancaster House and find a cross-party majority which will be along the lines I have indicated,” he said.