The Conservatives are under threat from “extreme” groups that have “welded themselves” to the party, a former Tory minister has warned.

Alistair Burt, who resigned as a Foreign Office minister on Monday in order to vote against the government on Brexit, said there were groups and think tanks that “band together” to try to influence the Conservatives.

Tories needed to be “very careful” about their impact, he said.

Mr Burt also said there needed to be a major “refresh” of the government once Britain leaves the EU.

Asked if he was concerned about the influence of hardliners in the party, the MP told The House magazine: “Not as members of parliament. Members of parliament have strong views and I’m not concerned about their influence.

“I am concerned about the influence of certain protest groups that have welded themselves to the Conservative Party, certain think tanks, the more extreme of those and the influence they seek to gain. Yeah, that’s a worry. The party has got to be very careful and guard against all this.”

He refused to be drawn on which organisations he was referring to but said some of the more extreme groups “band together”.

Mr Burt also said he was worried about reports of Islamophobia in the Conservative Party after a number of members and councillors were suspended over alleged anti-Muslim comments.

He said: “Have I been disturbed? Yes. Clearly, there is a recognition that of the Muslim community in the United Kingdom, many more support other parties other than the Conservative Party. As someone who wants to see the Conservative Party represent everyone in the UK, that is a matter of concern.”

How newspapers reported MPs seizing control of Brexit Show all 10 1 /10 How newspapers reported MPs seizing control of Brexit How newspapers reported MPs seizing control of Brexit The Independent How newspapers reported MPs seizing control of Brexit The Times How newspapers reported MPs seizing control of Brexit The Telegraph How newspapers reported MPs seizing control of Brexit The Mirror How newspapers reported MPs seizing control of Brexit The Metro How newspapers reported MPs seizing control of Brexit The I How newspapers reported MPs seizing control of Brexit The Guardian How newspapers reported MPs seizing control of Brexit The Financial Times How newspapers reported MPs seizing control of Brexit The Daily Mail How newspapers reported MPs seizing control of Brexit The Daily Express

The former minister, who backed Remain, added: “Once we get this [Brexit] stage over, the government has got to look towards an election in 2022. It would seem to me it’s a good time for a refresh of the government right the way through.

“Tactically, it’s a good time to do it, a good time for the country. Get some new people in place who will go forward. Not everybody, not going to have a wholesale change.

“But you are going to get some new faces. It’s a good time to do it.”

Theresa May announced this week that she will step down as prime minister if her Brexit deal is approved by parliament, firing the starting gun on the contest to succeed her.

Many Conservative MPs believe their next party leader should be someone who supported Brexit, but Mr Burt said he did not think this needed to be the case.

Support free-thinking journalism and attend Independent events

He said: “You look for the best person to do the job. You’ve got to look beyond did they vote Remain, did they vote Leave.”

He added: “I can understand the argument that someone who was a Brexiteer should take it forward, but again, I don’t think that’s exclusive.

“You could certainly have voted Remain and still be the right person to take us forward. It takes a tolerance from all colleagues to recognise though that they must put the polarisation … to one side when we get past the current votes and we’ve decided on what terms we’re leaving.