Talks between Labour and the government are unlikely to advance much further in the coming week unless Theresa May moves on her red lines over a future customs union, sources close to the talks have suggested.

David Lidington, who is leading the government’s talks with Labour, said a compromise would have to be reached but played down suggestions that a government shift was imminent and added that Labour would also have to move.

Labour has suggested the ball is in the government’s court and, while the opposition will engage on other topics including workers’ rights and security, the key question on customs arrangements remains unresolved. “She needs to take a political decision to move off her red lines – or not,” one source said.

Lidington said both sides had common ground on future customs arrangements but refused to say whether the government was prepared to agree to Jeremy Corbyn’s central demand for a common external tariff policy with the EU.

“We think it is possible to get the benefits of a customs union but still have the flexibility for the UK to pursue an independent trade policy on top of that with other countries outside the EU. Labour has a different approach,” he told the BBC’s Andrew Marr Show. “If we are going to reach an agreement there is going to have to be movement on both sides.”

The government has said three working groups will be established next week: security talks between the Brexit secretary, Steve Barclay, and his opposite number, Keir Starmer; environmental protections with Michael Gove and his shadow, Sue Hayman; and consumers’ and workers’ rights between the business secretary, Greg Clark, and his shadow, Rebecca Long-Bailey.

Sources suggested the chancellor, Philip Hammond, could meet the shadow chancellor, John McDonnell. And a meeting between party chiefs when MPs return to Westminster is being planned to assess the progress that had been made.

Lidington said the two sides “would hope to take stock of where we are as soon as parliament gets back after Easter recess but I don’t think this question can be allowed to drag out”.

He said there was “no date ringed in the calendar” for the talks to end but if agreement could not be reached on some form of Brexit deal then he hoped the two sides would be able to agree a binding mechanism for parliament to agree a way forward.

“What we want to do is agree a set of options with a system for making a choice with parliament actually having to come up with a preferred option rather than voting against everything,” he said.

May and Corbyn are not expected to be involved in the talks this week during the Easter recess, though Tory MPs expect speculation over the prime minister’s position and leadership jostling to continue. The former Tory leader Iain Duncan Smith said May should resign next month despite her failure to pass a withdrawal agreement.

“I know that the prime minister has already said she’s going. She said she would go as and when the agreement was ratified, which was looking at around about May, June. I think those dates still stand,” he told Sky’s Sophy Ridge on Sunday.

“I think that what the PM has to do is aim everything now towards departure before the Euros [elections], which would then allow her to step away having done what she said she would do, getting the UK out of the European Union one way or the other, and then we can have another leadership election and pick a new leader, which is the way it has to be.”

Two former chairs of the 1922 Committee of backbench Tory MPs, Michael Spicer and Archie Hamilton, said party rules could be changed to allow a leadership challenge sooner than December.

Writing in the Sunday Telegraph, the two peers said there was “nothing standing in their way” if MPs agreed to change the rules, though the committee’s current chair, Sir Graham Brady, said he was “less certain that it would be possible to change the rules during the current period of grace”.

However, the former party chairman Sir Patrick McLoughlin warned those agitating for a swift leadership change that installing a hard Brexiter in No 10 would not necessarily solve the Conservatives’ electoral problems.

“Defining ourselves as the Brexit party, pursuing the hardest form of Brexit with a parliament that will not deliver it, is a recipe for paralysis in government and suicide with the electorate,” he wrote for the Sunday Times. “We are the Conservative party, not the Vote Leave party.”

Nadine Dorries, a Eurosceptic MP and keen supporter of Boris Johnson’s leadership campaign, hit back at what she suggested was a concerted campaign against Johnson.

“There has been a stop Boris campaign since the days of Michael Howard pushing forward Cameron and Osborne,” she tweeted. “Boris is a big winner. Many of those with their own eye on No 10 aren’t a fan of that prospect.”

Duncan Smith said many in the party were deeply concerned about the most recent polling predicting a Labour lead of up to seven points and dire forecasts for the local and EU elections.

“It was on the 29th when we didn’t leave; that’s when this has all gone wrong. Up until then, people were prepared to give Theresa May the benefit of the doubt,” he said. “The big problem was as soon as we didn’t leave, you could see all the poll ratings start to crash.”