Rebel Conservative MPs have agreed to meet Jeremy Corbyn to discuss how to stop Boris Johnson pursuing a no-deal Brexit, without committing to backing him as a caretaker leader.

The Tory MPs Dominic Grieve, Caroline Spelman and Oliver Letwin, plus the former Conservative Nick Boles, said they would be willing to enter talks with the Labour leader in the weeks before parliament returns from recess.

“We agree that our common priority should be to work together in parliament to stop a no-deal Brexit and welcome your invitation to discuss the different ways this might be achieved,” they wrote.

Their tone was very different from that of Jo Swinson, the Lib Dem leader, who dismissed the idea of Corbyn leading a caretaker government as nonsense and said the Labour leader would not be able to build even a temporary consensus.

“This letter is just more red lines that are about him and his position and is not a serious attempt to find the right solution and build a consensus to stop a no-deal Brexit,” she said. “I am committed to working in a credible way with those in other parties, and none, across parliament to stop a no-deal Brexit and will set out how that could work in my speech tomorrow.”

In contrast, SNP, Plaid Cymru and Green MPs said they were willing to talk to Corbyn.

Following Swinson’s reply, Labour issued a direct plea to the Lib Dem leader to reconsider her refusal to work with Corbyn as a temporary prime minister in order to stop Boris Johnson pursuing a no-deal Brexit.

Play Video 1:09 Jeremy Corbyn urges MPs to back Labour plan to block no-deal Brexit – video

The shadow business secretary, Rebecca Long-Bailey, said it was sad that Swinson would not consider a caretaker government led by Corbyn and urged her not to shut the door on working together to stop no deal.

Speaking to BBC Radio 4’s Today programme, Long-Bailey said: “I’d issue a plea to Swinson particularly. I know that Jo wants to avoid a no-deal situation, as we do. This isn’t about personality or politics or about implementing Labour policy. It’s avoiding a no-deal situation and ensuring a general election is called so ultimately the people can decide which government they want to take forward … I think it’s sad Jo’s made those comments but wouldn’t close the door completely.”

Many Tory backbenchers and some independents are likely only to be willing to back a centrist candidate and not a current party leader for the job of leading a temporary government, if they can oust Boris Johnson by winning a vote of no confidence.

Anna Soubry, the leader of the Independent Group for Change (formerly Change UK), told BBC Radio 4’s Today programme on Thursday: “I would not support a government of national unity that is led by Jeremy Corbyn for all manner of reasons.

“One, because I don’t think it’s genuine, secondly because it is not going to deliver a people’s vote, which is the only way through the chaos; and, of course, he doesn’t command support or respect in his own political party, never mind across the parliamentary divide.”

The Totnes MP Sarah Wollaston, who announced she was joining the Lib Dems on Wednesday, said her new party leader was being realistic because Corbyn “won’t find the support of the Commons”.

She said none of the party leaders should lead a temporary government, and that a “trusted figure” such as Ken Clarke or Hilary Benn would be better.

UK could unilaterally exit EU in next 10 days, senior Tory MP says Read more

“I think it should probably be none of the party leaders that head that temporary arrangement, if that’s where we get to,” she said. “I think having a trusted figure that commands cross-party support would be much better for that very temporary arrangement if, indeed, that is one of the ways we go forward.”

Wollaston, a former Tory MP who joined the Independent Group for Change when it was known as Change UK, said she chose to make the switch to the Lib Dems to help make an unequivocal case against no-deal Brexit.

“The more people that join the Liberal Democrats and are making that unequivocal case, I think, the better, and the most significant feedback I got over the European election was that people wanted to see a single unified force making the case against no deal,” she told the Today programme.

Wollaston said that while her constituents in Totnes had voted for her initially as a Conservative candidate, they had wanted a centrist MP.

“Many of my constituents have been rather horrified by the way the Conservative party have shifted to the right. And I was selected originally by a fully open postal primary, the first in the country, so I think that people do have a support across my constituency for that kind of centre-ground approach,” she said.

“The point is that we are at a time of national crisis now. If you’ve got such a great policy idea, why do you have to spend billions on making contingency plans for air freighting in medicines and food? It is not good enough, and people deserve a say on that.”

The Plaid Cymru Westminster leader, Liz Saville Roberts, welcomed Corbyn’s call to work together in a time-limited caretaker government, but said Labour had to back a second referendum. “Stop clinging to what you said in the past. We need to work together seriously. In that respect we need to put a referendum first,” she said.

The SNP Westminster leader, Ian Blackford, also said he believed there was a majority in parliament against a no-deal Brexit. “I think really what is important is we tackle the immediate crisis in front of us, and that is the very real threat we have of leaving the European Union on a no-deal basis at the end of October, and I do believe, I strongly believe that there is a majority in parliament against no deal.”

He said he had written to Jeremy Corbyn asking for a meeting between all opposition leaders to put a broad coalition together to focus on opposing no deal.

“I believe that when we get back to parliament in the first week of September that we can bring forward legislation, we can do it through a mechanism called a SO24 application to stop no deal. And that is what we should be focusing on,” he said.

“It’s not about, in this immediate case, who is prime minister, it is about stopping that act of economic self-harm that all of us would suffer from.”