WASHINGTON ― As Republicans finish their long-sought rewrite of the tax code, some lawmakers are beginning to wonder just how much longer Rep. Paul Ryan intends to remain speaker.

“There’s a whole lot of rumors and speculation that the speaker may step aside,” one GOP member told HuffPost this week, a sentiment that was expressed by a number of Republicans who, perhaps tellingly, wouldn’t go on the record to speak about Ryan’s future.

The Wisconsin Republican has made no indication he’s quitting any time soon, but the possibility that Ryan finishes the tax bill and decides he no longer wants to continue in Congress has begun to loom over internal Republican conversations.

When the House Freedom Caucus gathered Monday night, members spent part of their meeting discussing a theory circulating on Capitol Hill and among the downtown Christmas parties that Ryan may believe he’s harpooned his personal white whale of tax reform and decide he’s finished.

“Is it a Boehner-meeting-the-pope moment?” one Freedom Caucus member rhetorically asked HuffPost, referring to Ryan’s predecessor, John Boehner (R-Ohio), who hosted Pope Francis for a joint address to Congress in September 2015 and then announced his retirement the next morning.

Ryan refused to run for the speakership twice after Boehner announced his resignation, once just after Boehner’s decision and then again after Majority Leader Kevin McCarthy (R-Calif.) abruptly changed his mind and stepped out of the speaker’s race.

Al Drago via Getty Images

Eventually, Ryan accepted the position, reluctantly, with some conditions. But lawmakers and aides who spoke to HuffPost on the condition of anonymity over the last week say they believe this is still a job Ryan doesn’t enjoy. On top of ruling over a GOP conference that’s just as unruly as when Boehner was speaker, Ryan has had to navigate Donald Trump ― a task that’s left him looking feckless to the left and like an enemy of the president to the right.

The speculation over Ryan’s next move has particularly intensified as Republicans negotiate spending deals with Democrats. Ryan has repeatedly pushed off the possibility that a legislative solution for the Deferred Action for Childhood Arrivals (DACA) immigration program will be attached to a government spending agreement, but conservatives are worried Republicans could finish their tax bill, have the speaker announce his retirement and then watch Ryan do the same kind of “barn cleaning” that Boehner did at the end of his speakership.

The last deal to raise spending caps for two years, in fact, was set under Boehner. Republicans and Democrats are reportedly close to a deal now to once again raise those spending limits for another two years, but conservatives are becoming increasingly certain this isn’t a bill they’ll support.

Ryan’s leaving could pave the way for Republicans to swallow a January spending deal with mostly Democratic votes, perhaps raise the debt ceiling again ― another thing Boehner did as he headed for the exit ― and potentially find a DACA fix.

It’s unclear whether Ryan has any further political aspirations beyond this job, but some Republicans think Ryan would be served well by offering himself as a sacrifice for the completion of an immigration deal, particularly if Ryan’s political aspirations are far off in the future. (Or if he doesn’t have any future political aspirations.)

On the other side of the argument, Ryan’s stepping down now could hurt Republicans. Ryan’s fundraising prowess has helped GOP campaign accounts stay flush with cash in an upcoming election cycle that could be tough for Republicans. And Ryan doesn’t seem to have any direct threat to his speakership. Nobody, not even the Freedom Caucus, is really discussing a motion to remove the speaker at this moment.

Ryan is also headed for a moment of coronation among GOP members with the passage of this tax bill. That could be the shot in the arm.

But that celebration will be short-lived if, as expected, upcoming spending deals put Republicans in the position of significantly raising non-defense spending and shoring up Obamacare.

Part of the problem for Ryan is simply that the expectations of the speakership have become unrealistic. The GOP conference has diffuse interests, and prioritizing those interests ― whether they be of defense hawks, fiscal hawks, moderates or just the Republicans who don’t shut down government ― will naturally create enemies.

Ryan isn’t without blame, of course. He managed to offend all sides with his on-again-off-again relationship with Trump, and he’s been less than honest about how he’s ruled over the House.

The 10-term congressman came into the speakership promising a more open legislative process. Two years into the job, however, Ryan has taken the restrictions on what amendments will be allowed on the floor to new heights.

There wasn’t a single “open” rule this year where lawmakers could offer any relevant amendment to a bill under consideration. And the legislative accomplishments Ryan touts ― whether it be the number of bills passed or that Republicans passed all 12 appropriations bills ― are grossly overstated. (The House has passed far more post office naming bills than in recent years, and the “12 appropriations bills” were in the form of a couple of packages, written to spending numbers that weren’t real, with a confined amendment process.)

Still, when HuffPost asked Ryan’s spokeswoman, AshLee Strong, whether the speaker believed he had lived up to his promises of an open process, she said he had.

Strong also said Ryan was committed to finishing this term in Congress and that he had no intention of retiring in the middle of a session.

Asked what else Ryan wanted to accomplish as speaker, Strong mentioned an infrastructure package and said “welfare reform” remained an “important unfinished item” from the GOP’s Better Way agenda.

That may sound definitive and convincing, but then again so did the quotes coming from Boehner’s staff the day before he announced his resignation.

When Time asked Boehner’s communications director, Kevin Smith, whether he was considering retirement, Boehner’s long-time aide was emphatic.

“He’s not going anywhere,” Smith said in a story that posted the morning Boehner announced his resignation. “If there’s a small crew of members who think that he’s just going to pick up and resign in the middle of his term, they are going to be sadly mistaken.”