Washington (CNN) One of President Donald Trump's friends said he believes the President is considering dismissing special counsel Robert Mueller, who was appointed to lead the FBI investigation into Russia's potential ties to the 2016 election.

"I think it is a consideration the President has had because Mueller is illegitimate as special counsel," Christopher Ruddy, the CEO of Newsmax Media, told CNN's Chris Cuomo on "New Day" Tuesday. "Chris, remember there is no evidence of wrongdoing, there's no evidence of collusion, there's no evidence of obstruction."

He added that he believes it would be a mistake to fire Mueller, but said the "the basis of his investigation is flim-flam."

Ruddy first made the bombshell assertion that Trump was considering dismissing Mueller to PBS' Jody Woodruff on "PBS Newshour" Monday night.

Chris Ruddy to @JudyWoodruff : President Trump is considering firing special counsel Robert Mueller, who he considered for another position. pic.twitter.com/X4IIHlh8at

But a source close to the President said Trump is being counseled to steer clear of such a dramatic move like firing the special counsel.

"He is being advised by many people not to do it," the source said.

And a person familiar with Trump's thinking said Tuesday morning that it's "unlikely" the President will fire Mueller, but conceded that it's often difficult to predict Trump's behavior.

"No one is recommending that," the person said, noting, however, that the option hasn't been ruled out.

Regarding Ruddy's comments, White House press secretary Sean Spicer said: "Mr. Ruddy never spoke to the President regarding this issue. With respect to this subject, only the President or his attorneys are authorized to comment."

And deputy White House press secretary Sarah Huckabee Sanders said simply: "Chris speaks for himself."

Tuesday morning, in a statement to CNN, Ruddy disputed Spicer's comments.

"Spicer issued a bizarre late night press release that a) doesn't deny my claim the President is considering firing Mueller and b) says I didn't speak to the President about the matter -- when I never claimed to have done so. Memo to Sean: focus your efforts on exposing the flim-flam Russian allegations against POTUS and highlighting his remarkable achievements! Don't waste time trying to undermine one of your few allies."

Ruddy also told CNN Spicer called him last night to ask him to issue a statement saying he didn't speak with Trump personally, but he refused. Asked to respond, Spicer said he called "and asked him to clarify that his statement was not based on any meetings or conversations with the president."

JUST WATCHED Newsmax CEO: Trump considering firing Mueller Replay More Videos ... MUST WATCH Newsmax CEO: Trump considering firing Mueller 01:37

Reports met with surprise

Mueller was appointed FBI director by President George W. Bush in 2001 and served until 2013, when Comey took over as head.

Still not everyone is a fan.

Earlier this week, former House Speaker Newt Gingrich reportedly told radio host John Catsimatidis that Congress should "abolish the independent counsel."

"I think Congress should now intervene and they should abolish the independent counsel," the former House speaker said. "Because Comey makes so clear that it's the poison fruit of a deliberate manipulation by the FBI director leaking to The New York Times, deliberately set up this particular situation. It's very sick."

Sen. Lindsey Graham, a South Carolina Republican, disputed that report.

"I don't think Newt said that," Graham told reporters. "I think it'd be a disaster. There's no reason to fire Mueller. What had he done to be fired?"

After news of Ruddy's interview surfaced on the web, Rep. Adam Schiff, the top Democrat on the House intelligence committee, echoed that sentiment on Twitter.

If President fired Bob Mueller, Congress would immediately re-establish independent counsel and appoint Bob Mueller. Don't waste our time. — Adam Schiff (@RepAdamSchiff) June 12, 2017

"If President fired Bob Mueller, Congress would immediately re-establish independent counsel and appoint Bob Mueller," the California lawmaker tweeted. "Don't waste our time."

Schiff later told CNN's Anderson Cooper he wouldn't be surprised if Trump was considering ousting Mueller.

"You have to hope that common sense would prevail," Schiff said. "But it wouldn't surprise me at all, even though it would be absolutely astonishing were he (Trump) to entertain this. The echoes of Watergate are getting louder and louder."

House Speaker Paul Ryan, however, did say he would be "surprised" if Trump fired Mueller.

"I think he should let Bob Mueller do his job, do his job independently, and do his job quickly, because I think that that's what he would want to have happen," Ryan told conservative commentator Guy Benson.

'Full independence'

Rosenstein told senators at a Senate appropriations subcommittee hearing Tuesday that Mueller will have the "full independence he needs to conduct that investigation."

He noted that Mueller can "only be fired for good cause," and it would be his job to put that good cause into writing.

"The chain of command for the special counsel is only directly to the attorney general -- and in this case, the acting attorney general," he said.

Rosenstein said "no, I have not," when asked by Sen. Jeanne Shaheen, a New Hampshire Democrat, whether he has seen good cause to fire Mueller.

"If there were good cause, I would consider it. If there were not good cause, it would not matter to me what anybody said," he said.