After a Freedom Caucus firebrand who helped initiate the ouster of former House Speaker John Boehner lost to a more mainstream candidate in his Republican primary on Tuesday, current House Speaker Paul Ryan is facing the first signs of his own leadership fight come the winter.

Tea Party mainstay and Kansas Rep. Tim Huelskamp lost his primary to a more moderate Republican candidate, an obstetrician named Roger Marshall who earned $3 million in backing from outside groups like the U.S. Chamber of Commerce and is now the presumptive favorite to win the seat in the fall. House conservatives are viewing the loss as a betrayal of their movement, and they are placing the blame squarely in one place: on House leadership, including Ryan.

“Clearly the war continues,” Huelskamp told CNN. “The establishment never sleeps. They spend more time going after conservatives than going after Hillary Clinton. It was about sending a message, and the message is this: ‘They came after me to get a scalp.’ “

“How the speaker responds to this in the next week or two will have major implications in terms of future speaker races,” another conservative said to CNN.

An anonymous Freedom Caucus member, meanwhile, told Politico that “how [Ryan] handles the aftermath of Huelskamp’s defeat will influence who the lawmaker supports for speaker next year.”

“[T]he House Republican leadership’s opposition to Tim Huelskamp significantly damaged the ability of House Republicans to [defeat Clinton],” House Freedom Caucus Chairman Jim Jordan, R-Ohio, said in his own statement.

Politico also reported that Ryan was facing direct blame from the Freedom Caucus for not doing anything to save Huelskamp’s hide:

The bloc of hard-line conservatives is laying the blame for Rep. Tim Huelskamp’s landslide loss squarely at House leadership’s feet. They’re furious that Ryan did not do more to help the Kansas Republican …

“He gave me no help pretty clearly,” Huelskamp said of Ryan.

The trouble for Huelskamp started in 2012 when he was removed from his position on the House Agriculture Committee, a critical spot for a representative from the farm state of Kansas. House leadership disregarded the notion that it had done anything to harm Huelskamp, but his supporters took issue with Ryan declining a request by the Freedom Caucus to promise that Huelskamp would be returned to the committee in the next term. Marshall focused much of his campaign on Kansas’ lack of representation on this crucial committee and it seemed to work—he won by more than double digits.

“Getting kicked off the Agriculture Committee is a crime that can’t be forgiven,” Brian Scheideman, a Kansas voter who cast his ballot against Huelskamp, told the Associated Press. “I don’t mind the independent voice, but you’ve got to figure out how to work with people.”

At the time of his ouster from that committee, Huelskamp blamed House leadership, telling Dave Weigel in a story in Slate: “It confirms, in my mind, Americans’ deepest suspicions about Washington. It’s petty, it’s vindictive, and if you have any conservative principles, you will be punished for it.”

The removal of Huelskamp, an influential member of the conservative Freedom Caucus, was one of the apparent motivations for the group’s coup against Boehner that led to his resignation last year. And Huelskamp opposed Boehner’s initial option for successor, Kevin McCarthy, because McCarthy refused to make a public statement promising to push back against the Chamber of Commerce and other groups that might oppose in primaries conservative incumbents who had quarreled with party leadership.

As Jim Newell wrote in Slate at the time: “It takes some chutzpah for Tea Partiers—who live to bring down ‘establishment’ candidates in primaries—to demand that they be shielded from counter-challenges themselves.”

Chutzpah or not, Ryan—who took the mantel of speaker after McCarthy went down in flames following some poorly chosen truthful words about Benghazi—did not ultimately come to Huelskamp’s rescue with the Chamber of Commerce and is now being blamed for that as well.

“When the Chamber comes in with as much outside money as they did, and the Chamber consistently gives to the speaker and members the speaker supports, to imply that they have no ability to push back on anything and no influence in how the Chamber spends its money defies logic and would be politically naive,” North Carolina Rep. Mark Meadows told Politico.

It seems practically impossible that Ryan could really face an uprising that would unseat him after just one term in office, but he could face an embarrassing number of “no” votes in the next speaker election that could sow the seeds for a future full-scale rebellion. This has happened before.

Meanwhile, Boehner had his own extraordinarily satisfied and satisfying response to Huelskamp’s loss:

Turns out Speaker Boehner is indeed enjoying Kansas primary night. Just received this from former staffer! pic.twitter.com/hIzmFt8hn0 — carl hulse (@hillhulse) August 3, 2016

Read more Slate coverage of the 2016 campaign.