One of the most influential families in GOP politics could rescind its longtime support of Rep. Justin Amash after the Michigan Republican said that President Donald Trump had committed impeachable offenses, a source close to the family said Monday.

The wealthy DeVos clan, which hails from Amash’s west-central Michigan district, backed the libertarian congressman in 2014 when he faced a primary challenger fueled by establishment forces in the party. Members of the family—which is one of the biggest financial supporters of conservative causes—collectively contributed $65,000 to Amash’s primary and general-election campaigns, helping him defeat his Republican opponent by 14 percentage points. But that could change with Amash facing a new primary challenge because of his Trump remarks, according to Greg McNeilly, a Michigan GOP operative who has long advised businessman Dick DeVos and his wife, Secretary of Education Betsy DeVos, on politics.

“I get the sense it’s been the straw that’s broke the camel’s back,” he said of Amash’s impeachment remarks.

A representative for the DeVos family, which has grown closely allied with the president, did not return a request for comment. McNeilly, who is a plugged-in GOP operative in the state, said he was not speaking on the family’s behalf. But his take on the state of play around Amash, he said, reflected the feelings held by many in conservative circles in western Michigan. McNeilly told The Daily Beast that emails, calls, and texts about Amash have been swirling since the congressman made his impeachment remarks. If a strong primary challenger to Amash were to run, he said, that candidate could attract funding from prominent conservatives in the area.

“People are fed up and done with him,” he said.

For Republicans already wary of speaking out against Trump, the backlash Amash has faced since Saturday provides additional proof of the dangers in doing so. It’s not just the DeVos family who has rallied around Trump in this latest tiff, but the conservative movement and its accompanying institutions as well. In a statement to The Daily Beast on Tuesday, Club for Growth President David McIntosh said Amash is “absolutely wrong on the standard for impeachment.” The Club, a conservative 501(c)(4) political advocacy group, spent more than $218,000 on ads and other communications to support Amash’s 2014 bid.

“In spite of his excellent voting record on economic issues, we completely disagree with him on this,” said McIntosh, though he did not say whether the group would refuse to support him in 2020.

By Sunday, a challenger to Amash in Michigan’s 3rd District had materialized. State Rep. Jim Lower, branding himself as a loyally pro-Trump conservative, announced his bid by claiming Amash’s remarks “show how out of touch he is with the truth and how out of touch he is with people he represents.” More challengers may yet enter the race. McNeilly said that recruitment efforts had been “supercharged” in recent days.

Officials at both the National Republican Congressional Committee, the House GOP’s official campaign arm, and the Republican National Committee, told The Daily Beast that they would not get involved in GOP primaries; though the RNC could change course if state committee members voted to do so.

But within Trumpworld, there is eagerness to see Amash punished. The congressman has been a subject of the White House’s scorn for years, owing to his penchant for voicing his aversion to certain Trump administration policies and voting against its proposed legislation.

Early on, Trump aides wrote off Amash as a wholly unreliable lawmaker within the otherwise friendly House Freedom Caucus. And over the past two years, senior Trump aides have discussed—on and off—the possibility of backing a primary challenger to Amash, one current White House official and one former official said. According to the current official, President Trump was in the room for at least one of these discussions.

Among those who wanted Amash gone was former White House chief strategist Steve Bannon, according to the former official. Bannon confirmed to The Daily Beast that he expressed his support for a primary during the early months of the Trump presidency, saying that Amash was a problem for the Trump White House “from day 1,” and that “for unity I thought he should be the only [GOP] primary of” the 2018 midterms.

At the time, Amash was publicly critical of the party’s efforts to nuke the Affordable Care Act, arguing that the initial ideas for a replacement didn’t actually repeal and replace Obamacare. That same year, Trump’s social-media director and close adviser Dan Scavino tweeted that the president was “bringing auto plants & jobs back to Michigan. @justinamash is a big liability. #TrumpTrain, defeat him in primary.”

After all that hoopla, the president didn’t throw his weight behind a primary challenger to Amash and the congressman won re-election easily.

White House Press Secretary Sarah Huckabee Sanders didn’t have any comment to add for this story, but referred The Daily Beast to the president’s recent statements on the matter.

“Never a fan of @justinamash,” Trump tweeted Sunday morning, adding that the congressman was a “total lightweight who opposes me and some of our great Republican ideas and policies,” as well as a “loser who sadly plays right into our opponents hands!”

Amash has appeared unfazed by the blowback he’s received. On Monday, he countered his critics in another Twitter thread, going after arguments that his Republican colleagues have made to defend Trump’s conduct.

The congressman’s office did not respond to request for comment. But his supporters say he isn’t losing too much sleep over Trumpworld’s public shaming. A source close to Amash said the lawmaker believes that most of the GOP voters in Grand Rapids, the largest city in his district, aren’t actually all that enthralled by Trump, and that any true primary threat to him wouldn’t be from a Trumpist conservative.

It’s a view shared by Amash’s circle of advisers and loyalists. “The numbers show that Justin’s district is not very Trumpy, and this is especially true in Kent County. A Trumpy Republican would probably lose to a Democrat,” Corie Whalen, Amash’s former communications director on Capitol Hill, told The Daily Beast. Still, the congressman’s district, which leans Republican by an average of 6 percentage points per the Cook Political Report, went for Trump by 10 points in 2016 and was the site of Trump’s final rally during that campaign.

“Justin is who he’s always been, and the idea that he’s changed in any way is absolutely false,” Whalen added. “The people who are angry with him should be asking themselves why they’re taking what he’s doing so personally.”

Amash’s decision to speak out on Trump’s potentially impeachable actions prompted several people in Michigan politics and conservative activism to speculate that he has his eye on a long-shot presidential bid in 2020, possibly under the Libertarian Party banner. But Amash supporters and critics alike say that there may be no long game, and that this provocative moment is just par for the course for someone who has made a name for himself by bucking his party.

“He is disillusioned, he is frustrated. He thinks Trump is the antithesis of libertarianism and limited government,” said a conservative strategist who knows Amash.

But asked if he thought conservatives would come to Amash’s defense, the operative said: “There’s nothing to defend… I do respect him and appreciate him. But I don’t know what he plans to get out of this.”

—With additional reporting by Lachlan Markay