(CNN) Maine Gov. Paul LePage declared Wednesday that he "will not resign," one day after he stoked speculation that he would leave office following the release last week of a profanity-laced voicemail that he left.

LePage held a news conference in his office Wednesday morning, and made clear that he would stay on as governor despite an avalanche of criticism for the offensive voicemail.

"Being called a racist was a horrible thing to me -- it was enormously hurtful. It hurt my family. I will not resign, though. My family (is) behind me," he said.

A day earlier, LePage had signaled in a radio interview that he was considering "all options" and suggested that "maybe it's time to move on" as criticism mounted.

"I am looking at all options. I think some things I've been asked to do are beyond my ability. I'm not going to say that I'm not going to finish it. I'm not saying that I am going to finish it," LePage told the the talk radio station WVOM in Bangor. "If I've lost my ability to help Maine people, maybe it's time to move on."

The Portland Press Herald first reported his comments. When asked for a response, LePage's press secretary encouraged reporters to listen to the entirety of the interview.

LePage later tweeted , "Regarding rumors of resignation, to paraphrase Mark Twain: 'The reports of my political demise are greatly exaggerated.' #mepolitics."

Regarding rumors of resignation, to paraphrase Mark Twain: "The reports of my political demise are greatly exaggerated." #mepolitics — Paul R. LePage (@Governor_LePage) August 30, 2016

LePage has been mired in controversy throughout much of his governorship, no more so than last week, when a voicemail he left for a state representative was made public . In the voicemail, the GOP governor berates Democratic State Rep. Drew Gattine with expletives and threats, apparently in response to Gattine calling him a racist -- something Gattine denied saying in an interview with CNN. "It's not something that I've ever called anybody," he said.

In the interview with WMOV, LePage said he takes full responsibility for the fallout from the voicemail -- "it's totally my fault" -- but that he was particularly enraged by the accusation of racism.

"When I was called a racist I just lost it, and there's no excuse," he said. It felt "like calling a black man the 'N' word or a woman the 'C' word. It just absolutely knocked me off my feet."

LePage said he met with Republican members of the Maine House Monday who want to "salvage what we can and move forward," but that the Republicans in the state's senate are "making demands."

"It's not about me," the governor said Tuesday. "It's about making sure that we can move the state forward. It's one thing to have one party behind, it's another thing to not have any party behind you."

The voicemail at the center of the controversy contained profanity and threats of violence, and prompted quick condemnation.

"Mr. Gattine, this is Gov. Paul Richard LePage," the audio recording says. "I would like to talk to you about your comments about my being a racist, you (obscene term). I want to talk to you. I want you to prove that I'm a racist. I've spent my life helping black people and you little (obscene term), socialist (obscene term). You -- I need you to -- just friggin'. I want you to record this and make it public because I am after you. Thank you."

After the voicemail leaked last Thursday, the governor doubled down and took credit for it in a later interview at his home with local media, and said he would like to challenge Gattine to a duel.

"When a snot-nosed little guy from Westbrook calls me a racist, now I'd like him to come up here because, tell you right now, I wish it were 1825," LePage said, according to the Press Herald. "And we would have a duel, that's how angry I am, and I would not put my gun in the air, I guarantee you, I would not be (Alexander) Hamilton. I would point it right between his eyes, because he is a snot-nosed little runt and he has not done a damn thing since he's been in this Legislature to help move the state forward."

Maine Democrats have called LePage "unfit to lead our state" and criticized him for what they saw as inciting violence.

LePage ultimately apologized for the voicemail in a statement last Friday, but justified his reaction by again saying the label of racist is "the absolute worst, most vile thing you can call a person."

"I didn't know Drew Gattine from a hole in the wall until yesterday," LePage said. "It made me enormously angry when a TV reporter asked me for my reaction about Gattine calling me a racist. ... So I called Gattine and used the worst word I could think of. I apologize for that to the people of Maine, but I make no apology for trying to end the drug epidemic that is ravaging our state."