The chairman of the House Tea Party Caucus and one of the first men to come out against Speaker John A. Boehner told Breitbart News that he does not think Speaker Paul D. Ryan (R.-Wis.) has the votes to keep his gavel.

“I just don’t think he has the votes,” said Rep. Timothy A. Huelskamp (R.-Kansas), who lost his primary Aug. 2 and returns to private life after this session of Congress.

Huelskamp said he is concerned that the House Republican leadership is in a hurry to, first, get itself rubber-stamped inside the conference and then to proceed with an aggressive lame duck agenda.

Many House conservatives have called for the leadership elections to be moved forward from the scheduled Nov. 15 event, he said.

It is important to watch what the GOP leadership does before returning Ryan, Majority Leader Kevin McCarthy, Majority Whip Steve Scalise, and Republican Conference Chairwoman Cathy McMorris Rodgers to their posts, he said.

The conference elections are by voice vote, which masks the weakness of Ryan’s support with House Republicans and could be chaotic if and when he fails to reach the required 218 votes when the new Congress meets.

“There is no doubt Ryan has the support of the majority of the conference, but we will not find out if he has enough until January,” he said.

“You got to have 218,” the congressman said. “Just because you have 215 doesn’t get you 218.

Huelskamp said he does not know who could possibly emerge as a rival to Ryan, but that also does not matter because the House rules do not require an opponent for someone to lose; all that is required is the 218 threshold.

“I was surprised over the years that when Boehner was on the ropes, no one would step forward,” the congressman said. “Paul Ryan? He wanted it and he got it.”

It was also a surprise to Huelskamp that the chairman of the House Finance Committee, Rep. Jeb Hensarling (R.-Texas), a former chairman of the House GOP Conference, never made a run for it.

“Hensarling was always the guy that I thought would do it, could do it, and would have had the votes,” he said. “Hensarling loves his job, but he knows the leadership will never let him take on Dodd-Frank.” Dodd-Frank is the financial services reform legislation, Wall Street Reform and Consumer Protection Act, which also created the Consumer Financial Protection Bureau.

The Kansan said that, in the lame duck session itself, he expects to see two bills come up that no one is expecting—the Internet Sales Tax and the Trans-Pacific Partnership trade deal, negotiated by President Barack Obama with our Pacific trading partners.

Before the current recess, the speaker led House Republicans to believe that he would not push for the TPP deal, but Huelskamp said if you read between the lines, Ryan actually left the door open. “He said it was not good enough, it needs to be renegotiated–it needs this, it needs that–but, he did not say: ‘We are not going to do it.'”

For six years, Republicans have blocked Democratic attempts to pass an Internet Sales Tax that would circumvent the 1992 Supreme Court Quill Decision, which ruled that the Quill office supply company was not obligated to collect sales taxes for states other than the state in which it was located.

Huelskamp said there is an Internet Sales Tax bill all ready to go, written by the staff of the House Judiciary Committee and supported by the GOP leadership.

The tax bill is set to be sprung without warning, he said. “No discussion before we went home. No discussion of that at all, not on the agenda–sounds like a typical insider lame duck deal.”

For his own swansong, the congressman said he is ready to make a privileged motion forcing a vote on the impeachment of the commissioner of the Internal Revenue Service.

In September, Huelskamp and his fellow member of the House Freedom Caucus Rep. John Fleming (R.-La.), a candidate for the Senate, were ready to make the motion, but the night before they had planned to execute the plan he announced that they would wait until after the election. One of the reasons House Conservatives were counting on Huelskamp and Fleming was that both men were leaving the House at the end of the session and would be immune from retribution from the House Republican Leadership.

Because the motion is a privileged motion, it must be voted on at once without debate or amendments, but Capitol Hill sources have speculated to Breitbart News that leadership looks to table the motion either in a vote or decision by the presiding officer.

Huelskamp said it was not his decision to stand down, but he went along with it because other conservatives begged him not to go through with it owing to the extreme pressure they were receiving from the leadership.

“I was strongly discouraged by House Freedom Caucus members, who said, ‘Now is not the time–we’ve got an understanding with Paul Ryan and the rest of the GOP leadership.'”

It was all at the last minute, he said. “I thought it was a mistake not to do it, but I didn’t do it out of respect for those who have to deal with Paul Ryan.”

A part of him suspects that the real reason members of the House Freedom Caucus lost their nerve was that they were afraid they would win, he said.

Regardless of the deals made in September, Huelskamp said he is making the motion for impeachment, when Congress returns from its current recess and before Congress leaves for its Thanksgiving recess.

The congressman, who holds a PhD in agricultural legislation, said he has not decided if he will run for Congress again after his term ends. He is convinced he was right on the issues, but he could not overcome the $2 million from outside the district that overwhelmed the voters beyond his own ability to get his message through.

“I am not sure what I am going to do.”