Dominic Grieve has blamed a former Ukip opponent for orchestrating an insurgency of his local association which has plunged his future into doubt after he lost a confidence vote.

The remain-supporting Tory MP is facing deselection from his party after the Conservative association in his Beaconsfield constituency said it no longer had confidence in him at a “rowdy” meeting on Friday.

But the former attorney general, who lost the confidence vote by 182 to 131, has hit back against the defeat by rounding on his former Ukip foe for the coup.

Jon Conway, who received just 1,609 votes compared with Grieve’s 36,559 when he stood against him in 2017, denied the accusation. The former regional organiser for Vote Leave said he was the only ex-Ukip activist who had joined the association.

Grieve, considered a moderate within the Conservative party, has been at the forefront of efforts to ensure parliament has influence over how Brexit is managed.

Leading Brexiter Boris Johnson tweeted his support for Grieve on Saturday morning, writing: “Sad to hear about Dominic Grieve. We disagree about EU but he is a good man and a true Conservative.”

Meanwhile, the Conservative party chairman Brandon Lewis gave his backing to Grieve and said the association vote against him had no formal standing under party rules.

Grieve, who has been MP for the constituency since 1997, told Sky News: “At the meeting there were a very large number of people who had turned up … around 100 of whom I had certainly never seen or met before in my years as a member of parliament.

“And there is clear evidence that there was an orchestrated campaign by my Ukip opponent in 2017, who has since joined the association, with the express intention of trying to come along and defeating the motion.

“And they were successful in doing that. It was a slightly rowdy meeting, although the chair was able to keep it under reasonable control.

“But certainly it wasn’t the type of Conservative party meeting that I’m used to attending.”

He added: “Mr Conway stood against me in 2017. I understand he subsequently informed the association that he had left Ukip, was no longer interested in it and that he wanted to join the association.

“And he actually said he didn’t want to cause any trouble, he just wished to join as an ordinary member. As we’re a broad church, he was admitted.

“But the evidence, I think, is pretty plain that he organised both bringing members into the association and also organised preliminary meetings prior to this AGM taking place for the expressed single purpose of coming to defeat the motion of confidence in me.”

Grieve said the local association was “fully entitled to express its views” but added: “I’ve always known that Brexit was very divisive and I’m totally respectful of the views of those who disagree with me as to the way I’ve approached this particular crisis.

“The Conservative party, being a broad church, historically doesn’t seek to expel members of parliament for variations of view otherwise many of my colleagues who currently have good careers in parliament would never have been able to remain there.”

He pointed out there are 1,100 members of the association, meaning the majority were not able to vote at the meeting. Grieve also said he received emails from Tory-supporting constituents who are “appalled” by Brexit.

Grieve said he would be “very concerned” if the local association was expelled over insurgency fears as it was run by “good people” and “functioning”. However, he added: “I am sure that we need to pay some attention to the extent to which infiltration has taken place and the extent to which there may be members of the association who in fact are not there for legitimate reasons.

“That having been said, my intention is to carry on as before.”

Conway denied he was behind the campaign against Grieve. “I’m one person out of 183 people who voted no confidence in Dominic last night,” he told Sky News. “The fact is, I’ve been a Conservative voter and activist for 35 years, along with 3 million other people approximately. We flirted with Ukip because we thought it was the best way of landing Brexit.

“He rightly says that I let the Conservative association know that I wanted to come back to be a Conservative member or voter.”

Though he said he respected Grieve’s views, he is fundamentally opposed to his stance on Brexit. “He’s never really believed in the idea of it and he’s become very outspoken,” he explained. “But he’s the rebel, not us – 17 and a half million were the majority. And last night there was a majority who are not happy with the way he’s representing us.

“Nobody has a divine right to be leader of a country or a constituency without challenge for 20-odd years. And our whole point last night was to say that we don’t have confidence in his view and we would like to be able to consider somebody else.”

Asked whether it was Ukip entryism, he replied: “The truth is I am the one and only single activist from Ukip to have joined the Conservative party in Beaconsfield. That’s the fact.”

Residents in the well-to-do Buckinghamshire constituency were generally supportive of Grieve on Saturday.

Tom, a distinguished man in his 70s, said Grieve was an “extraordinary MP” and it was very sad that “a man like that should potentially be lost to the government or parliament because of blind prejudice by obsessive Brexiteers”.

He had joined the Conservatives six months ago “because I could see the danger that the lunatic fringe would displace Dominic Grieve, and it looks as though that could happen”. The vote of no confidence had been “ominous”, he said, and there had been “a bunch of loud-mouthed yobs [at the meeting] who could easily have been Ukip”.

Alan Corbett, 58, said Grieve was a “very good bloke”. If the MP was deselected by the local party, he added, he would definitely vote for him as an independent.

However, Steve, who was shopping at the town’s Waitrose, hadn’t heard about the vote of no confidence in Grieve and declared it “fucking brilliant” news.

The 67-year-old, who moved to Beaconsfield 20 years ago, said he had always voted for Grieve, but “the major parties stood on manifestos that promised to abide by the result, and he has continued to defy it. He’s not fit for office.”

“I am quite happy to see the back of him,” he added. “I went to the Conservative office myself to see if there was anything I could do to make that happen. That’s the best news I’ve had all day!”