House Speaker John Boehner on Wednesday warned his House Republican conference against dissent from GOP leadership – and threatened retaliation similar to that against the already-purged conservatives if anyone else breaks from his pack.

The Hill reports that, according to Kansas Republican Rep. Tim Huelskamp, Boehner used his weekly GOP conference meeting to “note that we [leadership] have punished four members, he claimed that it had nothing to do with their conservative ideology, but had to do with their voting patterns.”

Huelskamp also said Boehner threatened “there may be more folks that will be targeted” and that Boehner told the caucus that “‘we’re watching all your votes.'”

It’s unclear what criteria Boehner and GOP leadership plan to use in the weeks and months ahead. Boehner spokesman Michael Steel wouldn’t provide specifics on what criteria or votes on which the Speaker is “watching” conservative Republicans. Instead, when asked, Steel provided Breitbart News with the exact quote another Boehner spokesman gave two nights ago for a response to a completely different question. “The Steering Committee makes decisions based on a range of factors,” Steel said.

Huelskamp is one of the four conservatives purged from committee assignments from which he would be able to affect fiscal policy. He was pulled from the House Budget Committee and House Agriculture Committee. Michigan Republican Rep. Justin Amash was pulled from the Budget Committee too, and Reps. David Schweikert of Arizona and Walter Jones of North Carolina were removed from the House Financial Services Committee.

Huelskamp also addressed the GOP caucus on Wednesday and said he received “warm reception from some and silence from others.” He asked GOP leadership to publicly provide “that list of votes used in the Steering Committee to reward or punish members.”

As first reported by Breitbart News, GOP leadership – Boehner, Majority Leader Eric Cantor, and Whip Kevin McCarthy – used a secret criteria list to decide which members to cut. Officials in each of their offices refuse to discuss the details of the list on the record with press and they refuse to publish it, so it’s unclear what criteria they used. It’s also unclear which figures in leadership sparked this process and what specific roles each official involved played.

At first, GOP leadership tried to pin at least part of the blame on the chairmen of the respective committees from which they pulled each member – Budget Committee chairman Rep. Paul Ryan and incoming Financial Services Committee chairman Rep. Jeb Hensarling. While a spokesman for Ryan hasn’t returned multiple requests for comment on the purge, a spokeswoman for Hensarling on Wednesday told Breitbart News that her boss had nothing to do with the purge.

The story from House GOP leadership keeps changing. On Wednesday afternoon, BuzzFeed reported – citing anonymous leadership aides – that there were different loyalty issues with each of the members. That’s a sharp departure from what Boehner had just told his caucus in its weekly meeting and what GOP leadership aides have told press at the beginning of this controversy.

“Huelskamp and Amash have repeatedly been thorns in the side of leadership, and in some cases their colleagues — in fact, one aide pointed out that Huelskamp voted against Rep. Paul Ryan’s budget,” BuzzFeed wrote. “And Schweikert engendered significant anger amongst elements of leadership during his reelection bid against fellow Republican Ben Quayle. He is also widely considered by leadership to be the source of an embarrassing story this summer about Republican members’ drunken swimming session in the Sea of Galilee.”

“Schweikert, notably, was also ousted from Majority Whip Kevin McCarthy’s Whip Team last year because of concerns with his loyalty,” BuzzFeed added. “The Whip team is a key organ of leadership, designed to help round up support amongst Republicans for bills coming to the floor. A Republican aide also pointed to the fact that Rep. Walter Jones, a North Carolina Republican who has consistently broken with leadership over the war, was replaced on the Financial Services Committee by Rep. Mick Mulvaney, a protégé of Sen. Jim DeMint and one of the most conservative members of the House.”

What those leadership aides told BuzzFeed after widespread conservative backlash directly contradicts what leadership aides told Roll Call about the purge before they faced any criticism. “You want good things in Congress and to have a good career? Better play along nicely,” a leadership aide told Roll Call.