The summer of Speaker Paul Ryan’s sustained headache over gun legislation seems to only be getting hotter, longer and increasingly self-inflicted.

After Democrats staged a drama filled 26 hour protest over gun-control on the House floor two weeks ago, Ryan didn’t want to give in to the minority party’s unprecedented sit-in, so he tried to up the ante by offering an anti-terrorism bill.

Who can disagree with that?

Well, it seems the political gamble wrapped in an anti-terrorism package was too much for the insurgent Tea Party wing of the Republican Party, due to the inclusion of a NRA-backed gun proposal.

Rank and file members of House Freedom Caucus — about 40 of the least agreeable Republicans in the House - announced their opposition to Ryan’s effort, in part, because conservatives would rather see him punish Democrats like civil rights icon John Lewis (D-Ga.) who spearheaded the sit-in.

“I think the sit-in was one of the most disgraceful exhibitions of disrespect for the institution, for the rule of law, that the House of Representatives has ever seen,” Rep. Tom McClintock (R-Cal.) told The Daily Beast. “The Constitution provides for the House to punish members for disorderly behavior.”

Even in the wake of the killing of 49 people at the Orlando LGBT nightclub, Republican leaders in the House never planned to hold votes on anything gun-related. But Democrats unexpectedly forced the issue into the conversation in Washington this summer, and now Republicans are angry that Ryan actually conceded and put the issue – even a watered down, NRA backed version that most Democrats hate anyway – on the House agenda.

“Most of us that have been parents realize from day one, you don’t reward bad behavior,” said Rep. Mick Mulvaney (R-SC) — an outspoken member of the Freedom Caucus. “And when somebody violates all the rules that they, you know, said they would adhere to, and sets bad precedent for the future, it simply shows that if you act badly you can get what you want.”

Mulvaney and other conservatives are pushing their party leaders to come down hard on Democrats who have vowed to keep pressure up on the GOP until they relent and bring Democratic gun-control measures to the floor this summer.

“One of the ideas that came out is like the NFL or the NBA, when their players commit violations, is they get fined,” Mulvaney told reporters before Ryan and Majority Leader Kevin McCarthy met with the Parliamentarian to discuss the rules broken by Democrats. “And maybe that’s something we should consider, if for the future, I don’t think that to retroactively do something would be the right thing to do. But I think that for the future we should make it clear that the rules are there to be adhered to.”

But an unruly civil rights leader isn’t the only issue the Tea Party wing of the party has with the bill. For one, they say it upends the Constitutional guarantee of due process by allowing the government to delay gun purchases for three days for people suspected of terrorist activity as the government presents their case to a judge.

The Freedom Caucus also opposes a portion of the underlying terrorism bill that would set up “an office within the Department of Homeland Security to prioritize counter Islamist terrorist radicalization in the U.S.”

While the first complaint is predictable, the second is a jarring shift for a party that, just a few years ago would have frothed at the chance to set up such an office.

Now that measure is partly to blame for endangering Ryan’s effort, which has angered some more hawkish Republicans.

“What’s changed is you’ve had a convergence of interests between what I would call the far left, or the liberal left, and the libertarian right, and those two groups are essentially united on some counter terrorism measures,” Rep. Charlie Dent (R-Pa.), one of the last moderate Republicans, told The Daily Beast. “That’s what’s changed: The libertarian right has asserted itself on many of these national security, homeland security, counter terrorism policy matters in a way they haven’t before.”

Ryan still has a couple days to brush off his proposal and gather the votes needed to pass it, but without the Freedom Caucus on board he’d need Democrats to get it passed and that’s unlikely to happen.