The Cabinet Office minister, David Lidington, has urged the Conservatives to “come together in a spirit of mutual respect” as senior Tories continue to air their differences over Brexit.

Speaking after a series of open spats, Lidington, who works closely with the prime minister in Downing Street, told the BBC’s The Andrew Marr Show that all sides of the “broad church” should unite to confront Labour.

“I think what I’d say to all my colleagues is that the Conservative family – left, right and centre, because we’re a broad church – has to come together in a spirit of mutual respect,” he said.

“There are differences in any broad church, but look at what the bigger picture is showing. The bigger picture is showing that after eight years in government, we’re neck and neck with the Labour party in the polls.”

The latest confrontation over Brexit saw the former Northern Ireland secretary Theresa Villiers warn Britain was in danger of remaining in the EU “in all but name”.

Villiers, one of six cabinet ministers from David Cameron’s team who campaigned for leave in the 2016 EU referendum, used a Sunday Telegraph article to warn against a “dilution of Brexit”.

It comes after Jacob Rees-Mogg, the chair of the influential European Research Group of pro-Brexit MPs, accused the government of being “cowed” by Brussels last week.

Leavers are concerned about the nature of the implementation period after March 2019, which is expected to involve the UK accepting the jurisdiction of European court of justice, and they have been alarmed by reports that officials are seeking to extend the period beyond two years.

Quick guide What are Brexit options now? Four scenarios Show Hide Staying in the single market and customs union The UK could sign up to all the EU’s rules and regulations, staying in the single market – which provides free movement of goods, services and people – and the customs union, in which EU members agree tariffs on external states. Freedom of movement would continue and the UK would keep paying into the Brussels pot. We would continue to have unfettered access to EU trade, but the pledge to “take back control” of laws, borders and money would not have been fulfilled. This is an unlikely outcome and one that may be possible only by reversing the Brexit decision, after a second referendum or election. The Norway model Britain could follow Norway, which is in the single market, is subject to freedom of movement rules and pays a fee to Brussels – but is outside the customs union. That combination would tie Britain to EU regulations but allow it to sign trade deals of its own. A “Norway-minus” deal is more likely. That would see the UK leave the single market and customs union and end free movement of people. But Britain would align its rules and regulations with Brussels, hoping this would allow a greater degree of market access. The UK would still be subject to EU rules. The Canada deal A comprehensive trade deal like the one handed to Canada would help British traders, as it would lower or eliminate tariffs. But there would be little on offer for the UK services industry. It is a bad outcome for financial services. Such a deal would leave Britain free to diverge from EU rules and regulations but that in turn would lead to border checks and the rise of other “non-tariff barriers” to trade. It would leave Britain free to forge new trade deals with other nations. Many in Brussels see this as a likely outcome, based on Theresa May’s direction so far. No deal Britain leaves with no trade deal, meaning that all trade is governed by World Trade Organization rules. Tariffs would be high, queues at the border long and the Irish border issue severe. In the short term, British aircraft might be unable to fly to some European destinations. The UK would quickly need to establish bilateral agreements to deal with the consequences, but the country would be free to take whatever future direction it wishes. It may need to deregulate to attract international business – a very different future and a lot of disruption.

The chancellor, Philip Hammond, enraged pro-Brexit ministers and backbenchers last week by saying in a speech in Davos that Britain’s trading relationship with the EU would change only “very modestly” after Brexit. A backlash in Westminster later forced Downing Street to distance the prime minister from the remarks.

Villiers said she supports “compromise and moderation” in the UK’s approach to the negotiations, but added: “I understand why some are becoming nervous about the current situation.

“Since the prime minister set out a bold vision in her Lancaster House speech, the direction of travel seems to have gone in only one single direction: towards a dilution of Brexit.

“If the government goes too much further down that path, there is a real danger that it will sign up to an agreement which could keep us in the EU in all but name, and which would therefore fail to respect the referendum result.”

She urged May to resist the “immense pressure” to “water down” the approach set out last January, when the prime minister said Britain would be outside the single market and customs union.

May has so far largely managed to keep a lid on the divisions within her party over Brexit, but ministers privately admit there are deep differences at cabinet level about how much the UK should hope to diverge from the EU in future.

The foreign secretary, Boris Johnson, plans to deliver a speech on what he calls the liberal case for Brexit in the next few weeks, which could cause fresh discord if it appears to differ from the prime minister’s approach.

Johnson was rebuked by May and several cabinet ministers last week after allies briefed a series of newspapers that he planned to confront the prime minister to demand a funding increase for the NHS.

Play Video 1:29 Jeremy Corbyn on Brexit: 'We are not asking for a second referendum' – video

Jeremy Corbyn, also appearing on Marr on Sunday, denied that Labour’s position on Brexit was muddled. “What we wanted to do was say we’ll protect jobs and supply chains on both sides of the Channel,” he said.

Jeremy Corbyn convenes ‘away day’ to discuss Brexit policy shift Read more

“There has to be the closest possible trading relationship with Europe, there has to be a tariff-free access to European markets. That’s where we’re going to with the objective.”

Asked whether Labour would be willing to accept aligning its regulations with the EU to win such tariff-free access, he said: “We would have a regulatory environment that is commensurate with European levels of regulation – obviously, because half our trade is with Europe.

“The point has to be about the regulatory environment and above all, the ability to influence those regulations that come.”

But he insisted Labour would not want to remain within the existing customs union with the EU. “We would want a form of customs union, obviously; whether it would be the customs union, answer: no, because it would require being a member of the EU, which we’re not.

“We would need to be sure that any relationship with the EU gives us the chance to influence the situation we’re in and any trading relationships we want.”

A vocal group of Labour backbenchers is urging Corbyn and his team to advocate remaining inside the existing customs union.

The Labour leader also confirmed that his party did not plan to throw its weight behind Liberal Democrat calls for a second referendum. “We’re not asking for a second referendum,” he said. Asked “and you’re not going to?”, he replied: “No.”

