Speaker John Boehner (R-OH) displayed a rare flash of anger at outside conservative groups on Wednesday when asked about their opposition to a two-year bipartisan budget deal struck between Rep. Paul Ryan (R-WI) and Sen. Patty Murray (D-WA).

“You mean the groups that came out and opposed it before they ever saw it?” a visibly angry Boehner told reporters Wednesday, his tone rising. “They’re using our members and they’re using the American people for their own goals. This is ridiculous.”

Conservative groups Club For Growth, Heritage Action, FreedomWorks and Americans For Prosperity all began speaking out against the emerging deal before it was announced on Tuesday evening. These groups hold significant sway among House Republicans and have repeatedly thwarted Boehner’s attempts to govern and pass bills.

The conservative organizations griped that the emerging deal, the broad outlines of which had been reported by TPM and other news outlets, would raise spending from current levels of $967 billion to $1.012 trillion in 2014. The agreement would reduce the deficit by about $20 billion, and mitigate some of the across-the-board sequestration cuts by replacing them with targeted discretionary spending cuts and non-tax revenue.

“Listen, if you’re for more deficit reduction, you’re for this agreement,” Boehner said.

Boehner also attacked the outside conservative groups inside the closed-door meeting on Wednesday with House Republicans, a source in the room told TPM.

“These groups aren’t acting out of principle, and they’re not trying to enact conservative policies. They’re using you to raise money and expand their own organization,” Boehner said, according to the source’s paraphrase. “No one controls your voting card but you.”

This article has been updated.