Rep. Jason Chaffetz (R-Utah), who stunningly announced this week that he won’t seek re-election and may leave office before his term ends, strongly denied he’s quitting because of some as-yet undisclosed scandal.

“Not in any way shape or form,” Chaffetz, chair of the House Oversight Committee, said in an interview with Politico. “I’ve been given more enemas by more people over the last eight years than you can possibly imagine,” he continued. “From the Secret Service to the Democratic Party. I am who I am. If they had something really scandalous, it would’ve come out a long, long time ago.”

Chaffetz made the shock announcement this week that he won’t run for re-election next year, or be “a candidate for any office” in 2018. In the days since, he said he may not even finish out his current term.

“If I do step down early, it will be months from now,” Chaffetz told The Wall Street Journal. “It’s hard for people to believe, I really do want to be with my family more. Maybe it’s more a midlife crisis more than anything else.”

The announcement has prompted waves of speculation about Chaffetz’ future. Some have said he may be eyeing a bid for governor in 2020. Chaffetz told the Journal he was looking at the private sector.

Utah election officials told Reuters they haven’t been formally told about any plans for Chaffetz to resign. Mark Thomas, the state’s director of elections, told Reuters officials have begun gathering information about when and how to hold a special election, but “don’t know what he’s planning.”

“This is more informal, just wanting to know ourselves and be prepared,” Thomas said.

Chaffetz has drawn fierce criticism from Democrats in recent months over claims he’s been slow to investigate President Donald Trump’s ongoing and unfounded claims of voter fraud and wiretapping by former President Barack Obama. He also drew sharp rebukes from Republicans and constituents after saying in February he wouldn’t look into ties between former Trump national security adviser Michael Flynn and the Russian ambassador ― a relationship that led to the Flynn’s firing.

“It’s taking care of itself,” Chaffetz said at the time.