Tory MPs who oppose a hard Brexit should defect or resign rather than vote against their conscience, the Liberal Democrat leader, Tim Farron, has said in a direct appeal to Conservatives to join his party.

Farron said the Lib Dems had the momentum to defeat Tories who decide to remain loyal to Theresa May’s Brexit agenda, citing his party’s victory over the former Tory MP Zac Goldsmith in last year’s byelection in Richmond Park, a constituency that voted heavily to remain in the EU in the June 2016 referendum.

Addressing his party’s spring conference in York, Farron said Tory MPs should stop supporting policies in parliament that were the opposite of those they had signed up for. “You are now the supporters of a government that is as anti-business as Jeremy Corbyn,” he said. “You are now the cheerleaders of a government that is as anti-refugees as Nigel Farage.

“You know it’s wrong, so for pity’s sake, have some self-respect. Defect or resign. If you don’t, then when the next election comes we will do to you what we did to Zac Goldsmith.”

Farron said the Lib Dems were determined to take votes from the Tories’ pro-business and internationalist wings. “Theresa May has put at risk the very people who have bankrolled her party’s success for years, and she didn’t have to. So business should drop the Conservative party like a hot brick,” he said.

“We can gain the seats to rob the Tories of their power to wreck Britain, and by doing so we can change the course of our country,.”

Farron, whose party was reduced to just nine MPs in the 2015 general election after governing in coalition with the Conservatives, said he still aimed to replace Labour as the main opposition party, with Labour unwilling to block a hard Brexit.

The four most recent former prime ministers – David Cameron, Gordon Brown, Tony Blair and Sir John Major – had more in common with the Lib Dems than their own parties, he said.

“The Conservative party has been taken over by its own version of Momentum. May’s Momentum, the hard Brexiteers. The anti-free trade protectionists. The shrink-the-state extremists. The anti-immigrant, anti-refugee, anti-international aid zealots. It’s their party now, and it’s hard to be sure whether Theresa May is their leader or their captive.”

Farron also issued a call for moderate Labour MPs and other centrists not to form new parties but to join the Lib Dems. “Some of them might be hoping that the Labour party will sort itself out. I’ve got news for you, Labour are done,” he said.

“It is a huge opportunity for a progressive party. This party. All you generals without armies, here’s your army.”