Washington (CNN) President Donald Trump is openly wondering what happened to the collection of officials he described only five months ago as the "finest group of people ever assembled" as members of his Cabinet find themselves embroiled in ethics scandals or woefully at odds with the White House.

He has complained in recent days that his Cabinet has fallen well short of his expectations and wants to purge all the "deadweight," one official said.

A third, Education Secretary Betsy DeVos , caused alarm at the White House earlier this week when she appeared on television and wasn't able to answer basic questions about the country's schools. And two other Cabinet-level officials -- national security adviser H.R. McMaster and chief of staff John Kelly -- are both rumored to be heading toward the exits.

Trump's frustration with Shulkin appears to be the most strident. People familiar with the matter said Trump is making plans to remove Shulkin from his post after parallel events brought his agency into turmoil: an ethics scandal involving a trip to Europe with his wife and an internal insurrection from members of his own staff. Trump has eyed Energy Secretary Rick Perry as a possible replacement, a person familiar with the matter said, though it's far from clear whether Perry is interested in moving.

The White House declined to comment Wednesday on reports of an impending administration shake-up, saying only that when Trump decides to make changes he will announce them at the time.

"There's a lot of speculation. We deal with rumors and innuendo all the time. There are no personnel announcements," spokesman Raj Shah told reporters aboard Air Force One.

"The President has confidence in his entire team. When he does not, you guys wiIl know about it," Shah said.

The emails

On Wednesday, Carson's woes involving the purchase of a $31,000 furniture set for his office appeared to deepen, and his standing now appears in question, sources indicated. CNN first reported on Wednesday the existence of new emails that cast doubt on claims made by Carson and his spokesman that he had little or no involvement in the purchase of the dining set for his office.

An August email from a career administration staffer to Carson's assistant, with the subject line "Secretary's dining room set needed," refers to "printouts of the furniture the Secretary and Mrs. Carson picked out."

That contradicted earlier claims by the agency that Carson "had no awareness that the table was being purchased." Once reports emerged earlier this year about the cost of the dining set, Carson canceled the order.

The White House didn't respond on Wednesday when asked specifically about Trump's confidence in his Cabinet members, nor did representatives for Carson or Shulkin.

Trump is disturbed by the bad headlines involving Carson and can't believe he got himself into his current predicament, according to an administration official. Trump has made known across the Cabinet that he has no tolerance for ethics scandals, and the White House held private meetings with four Cabinet-level officials -- including Carson, Shulkin, Interior Secretary Ryan Zinke and Environmental Protection Agency Administrator Scott Pruitt -- last month to scold them for embarrassing stories about questionable ethical behavior at their respective agencies, sources familiar with the sessions tell CNN.

Internal watchdogs have launched at least nine audits, reviews or investigations across several Cabinet agencies, and stories about first-class travel, expensive office furniture and internal strife have become commonplace.

Ready to clean house

People inside the West Wing believe Trump is ready to clean house. Despite attempts to bring the Cabinet onto the same page -- including convening near-monthly Cabinet meetings -- Trump now views his Cabinet as burdensome and more likely to produce bad headlines than good ones.

Trump has aired some of his frustrations in public, including lambasting "beleaguered" Attorney General Jeff Sessions, whom he's never forgiven for recusing himself from Russia-related matters at the Justice Department. Before firing Tillerson, Trump voiced his differences with his top diplomat on Twitter.

He hasn't openly criticized the secretaries currently embattled in ethics or other scandals, but he is fed up, people who have discussed the matter with him say. He's also expressed confusion at how the group of people he selected himself could have wound up disappointing him.

It was only in October that Trump extolled the "tremendous amount of talent" in his administration during a Cabinet meeting at the White House: "We have just gotten really, really, great people. I'm very proud of them."

As always, however, the timing and precise details of any shake-up will be determined by Trump alone. After firing Tillerson on Tuesday, he hinted the dismissals weren't over.

JUST WATCHED Dr. Ben Carson addresses HUD qualifications Replay More Videos ... MUST WATCH Dr. Ben Carson addresses HUD qualifications 01:45

"We're getting very close to having the Cabinet and other things that I want," Trump told reporters on the South Lawn.

Trump remains pleased with the work some of his Cabinet officials are doing. Defense Secretary James Mattis continues to hold sway at the White House, despite sometimes disagreeing with the President on national security matters. Commerce Secretary Wilbur Ross helped shepherd through Trump's beloved steel tariffs, and even a mocked television appearance involving a can of chicken noodle soup didn't bother Trump. And Treasury Secretary Steve Mnuchin has remained influential, largely through sticking closely by the President's side.

Other officials who remain in Trump's good graces -- including Transportation Secretary Elaine Chao, Labor Secretary Alex Acosta and Agriculture Secretary Sonny Perdue -- have operated in relative obscurity. On Wednesday morning, Trump expressed confidence in those officials on Twitter.

Five of our incredible @Cabinet Secretaries are testifying on the Hill this morning on the need to rebuild our Nation's crumbling infrastructure. We need to build FAST & we need to build for our FUTURE. Thank you @SenateCommerce for hosting this hearing! #InfrastructureInAmerica — Donald J. Trump (@realDonaldTrump) March 14, 2018

"Five of our incredible @Cabinet Secretaries are testifying on the Hill this morning on the need to rebuild our Nation's crumbling infrastructure," he wrote. "We need to build FAST & we need to build for our FUTURE."