(CNN) White House officials are discussing plans to replace Health and Human Services Secretary Alex Azar following a spate of criticism over the early federal response to the coronavirus , a senior administration official confirmed.

Any move to replace Azar would be contingent on President Donald Trump deciding to move forward, and there is currently little appetite inside the White House for a big shake up amid the coronavirus pandemic. The official stressed that nothing is imminent, but there are discussions underway at the White House about replacing Azar.

The President tweeted Sunday evening that reports about him firing Azar are false and said, "Alex is doing an excellent job!" Shortly after, Azar quote tweeted the President's tweet as part of a multi-tweet thread. Like Trump, he smeared the press and called the reports that the President was looking to replace him "fake news," then lavished praise on Trump's handling of the pandemic.

Still, the discussions come after Trump appointed Vice President Mike Pence , rather than Azar, as the White House's point man for the coronavirus response at a rare presidential news conference. Trump had privately expressed frustration over Azar's lack of communication on key issues as the head of the White House coronavirus task force.

Azar has also been at odds with Centers for Medicare and Medicaid Services Administrator Seema Verma, an ally of Pence's from Indiana who has been involved in the coronavirus task force briefings.

As the coronavirus spreads across the nation, the Trump administration has been criticized for its handling of the pandemic, namely by governors, on both sides of the aisle, in desperate need of critical supplies for their states as the President has moved forward on indicating that many states can reopen by May 1.

In a statement Saturday night, deputy White House secretary Judd Deere said, "The Department of Health and Human Services, under the leadership of Secretary Azar, continues to lead on a number of the President's priorities. Any speculation about personnel is irresponsible and a distraction from our whole-of-government response to COVID-19."

The move to replace Azar was first reported by the Wall Street Journal and Politico

"Secretary Azar is busy responding to a global, public health crisis and doesn't have time for palace intrigue," HHS spokeswoman Caitlin Oakley said in response to the reports.

The discussions come amid two major shifts in top personnel at the HHS in recent weeks -- changes that follow months of turmoil at the agency.

Last week, former 2016 Trump campaign adviser Michael Caputo was named to serve as the agency's new spokesman. A source familiar with the situation told CNN that Caputo was brought in because of the President's distrust of Azar and to expect to see an uncomfortable power dynamic between the two men.

And earlier this week, the director of the agency's office involved in developing a coronavirus vaccine said he was abruptly dismissed from his post in part because he resisted efforts to widen the availability of a coronavirus treatment pushed by the President.

Dr. Rick Bright , who had led the Biomedical Advanced Research and Development Authority, had clashed directly with Azar, two people familiar with the situation told CNN. A senior HHS official denied that Bright and Azar clashed.

Azar's role in the Trump administration

As secretary, Azar has overseen several top administration priorities, including efforts to weaken the Affordable Care Act, combat the opioid crisis and reduce the cost of prescription drugs.

A senior Republican told CNN that discussions about possibly removing Azar are not surprising because he has been on thin ice for months -- partially for genuine problems handling the coronavirus pandemic as well as frustrating the President earlier for how he dealt with other issues, like Obamacare.

Another source who is close to Azar but critical of some of his actions said he believes the talk about possibly replacing him is coming from advisers close to the President who are using Azar as a scapegoat for White House missteps, including Trump's statements at the White House briefing this week regarding disinfectant and heat being able to kill the virus.

The second source said Azar is "a good guy," who definitely made mistakes but is also being used to cover for the President's own missteps and Trump's reluctance to move on from the pandemic because of his concerns about the economy. This source also said that the President was getting mixed advice on firing Azar during the pandemic crisis. Some people would like to replace him, but others said it would just look bad and add to the chaos, the source added.

Azar, who joined the Trump administration in January 2018, served as general counsel and deputy secretary of the agency under former President George W. Bush. He then joined Eli Lilly, becoming president of Lilly USA in 2012. He spent nearly nine years at the company, during which time it and other drug makers greatly hiked prices.

He was appointed Health and Human Services secretary after his predecessor Tom Price resigned over his use of private planes for government business trips.

An economist and lawyer by training, Azar clerked for Supreme Court Justice Antonin Scalia in the early 1990s and later worked under Kenneth Starr, the independent counsel on the Clinton Whitewater investigation who defended Trump during his impeachment trial.

This story has been updated Sunday with additional developments.

CORRECTION: This story has been updated to correct the number of coronavirus cases.