Media playback is unsupported on your device Media caption Sir Michael Fallon told Today the Tories should be careful not to drive Remainers away

The expulsion of 21 Conservative MPs for rebelling against the government over Brexit "sends the wrong message to Remainers", a party veteran has warned.

Sir Michael Fallon said the move by No 10 could drive voters to parties who support staying in the EU.

High-profile figures like former Chancellor Ken Clarke were among those to have the whip removed on Tuesday.

Sir Michael told BBC Radio 4's Today programme he hoped they would be able to appeal against the decision.

Asked if there could happen, Prime Minister Boris Johnson said he was "going to reach out to them [and] try to find ways of building bridges".

He added: "It grieved me deeply. These are friends of mine and I have worked with them for many years.

"But I have to get Brexit done and we were very clear in the risk of snarling up the process of leaving the EU in Parliament".

Media playback is unsupported on your device Media caption Boris Johnson: "It grieved me deeply"

An alliance of opposition MPs and Tory rebels won a vote in the Commons on Tuesday to take over the business of the House.

This gave them the opportunity to put forward a bill the next day to prevent a no-deal Brexit on 31 October - forcing the PM to ask the EU for an extension to the deadline if a deal is not agreed - and it was passed by 327 votes to 299.

Downing Street reacted by removing the party whip from Conservatives supporting the move, meaning they cannot stand as Tory candidates at the next election.

Several of the 21 have said they will now leave front line politics at the next election but others have said they will contest the decision to exclude them, Philip Hammond promising his party the "fight of a life time".

Antoinette Sandbach, the MP for Eddisbury in Cheshire, said on Friday she would be keeping her options open and was considering whether to appeal against the decision, to stand as an independent or potentially join another party.

She told the BBC she believed another referendum was now the only way forward to resolve the Brexit stalemate, fuelling speculation she could follow other former colleagues Sarah Wollaston and Phillip Lee in joining the Liberal Democrats.

Former Defence Secretary Sir Michael said the vote on the bill was "a matter of confidence" in the government, so removing the whip was the "proper process".

However, he said he hoped there was "some kind of appeal mechanism that they can find now so that they get the chance to state their case".

Sir Michael said there was a wider issue with the expulsion, though.

"I also worry that it sends the wrong message to Remainers, particularly in my party," he said.

"I think by definition some five million Conservatives must have voted Remain.

"I think we've got to be very careful not to drive them into the hands of Remainer parties like the Liberal Democrats in England, or the Scottish Nationalists in Scotland."

'There is still time'

Sir Michael said it was "very important that we take those who voted Remain - and nearly half the country voted Remain - we do try and take them with us."

Asked it if was too late, he said: "I hope it isn't. We have pledged to withdraw in an orderly way, there are seven weeks left to find an agreement and there are important talks taking place.

"There is still time to find a way through this with some sort of compromise agreement, and it is very important that those who voted Remain do feel they are wanted with us on this voyage into a new future."

Sir Michael, MP for Sevenoaks, Kent, since 1997, confirmed he would be stepping down at the next election, but denied it was linked to the events in his party this week.

"I was intending to stand down at the end of this Parliament in a few years' time... and I don't want to commit again for what could be another five years," he said.

Former energy minister Claire Perry has also announced she will not contest her Devizes seat at the next election, which she has held since 2010.

In a tweet revealing her decision, Ms Perry said she would continue to give the PM and his "brave" Brexit strategy her backing.

Following the vote on Tuesday, a Downing Street spokesperson said: "The chief whip is speaking to those Tory MPs who did not vote with the government this evening. They will have the Tory whip removed."