Former White House communications director Anthony Scaramucci Anthony ScaramucciFormer DeVos chief of staff joins anti-Trump group Scaramucci to Lemon: Trump 'doubling down' on downplaying virus 'should scare' viewers Sunday shows - Leaked audio of Trump's sister reverberates MORE said on Monday that he’s assembling a team of former Cabinet members to speak out against President Trump Donald John TrumpFederal prosecutor speaks out, says Barr 'has brought shame' on Justice Dept. Former Pence aide: White House staffers discussed Trump refusing to leave office Progressive group buys domain name of Trump's No. 1 Supreme Court pick MORE in an effort to find a Republican challenger to the president in 2020.

“I'm in the process of putting together a team of people that feel the exact same way that I do. This is not a ‘Never Trump’ situation, this is not just screeching rhetoric. This is, 'OK, the guy's unstable, everyone inside knows it, everyone outside knows it, let’s see if we can find a viable alternative,'” Scaramucci said Monday on CNN’s “New Day.”

“Moreover, I’ve got to get some of these former Cabinet officials in unity to speak up about it.” ADVERTISEMENT

Scaramucci would not reveal names of former officials that he said feel that Trump is unstable. But he said he expects more to come out publicly in upcoming months.

“I predict in middle or late fall there will be a trove of people that will come together in unity to say this is what’s going on. This is how the person’s acting. This is why there’s nobody inside the White House he’s taking any advice from,” Scaramucci said.

He also did not disclose names of possible candidates he’s looking to back along with the team he’s looking to assemble.

“I don’t think it’s fair to those people,” he said.

Around the same time Scaramucci spoke on CNN, Trump renewed an attack on his former staffer. He tweeted that Scaramucci is a “nut job” that he “barely knew.”

“He was a mental wreck. We didn’t want him around. Now Fake News puts him on like he was my buddy!" Trump tweeted.

Trump also tweeted that there is "great cohesion" inside the Republican Party.

Scaramucci had long defended Trump but in recent weeks has spoken out against the president’s rhetoric and actions.

Trump has drawn one official Republican primary challenger so far, former Massachusetts Gov. Bill Weld William (Bill) WeldRalph Gants, chief justice of Massachusetts supreme court, dies at 65 The Hill's Campaign Report: Biden visits Kenosha | Trump's double-voting suggestion draws fire | Facebook clamps down on election ads Biden picks up endorsements from nearly 100 Republicans MORE.

Weld previously appeared on the Libertarian Party presidential ticket with former New Mexico Gov. Gary Johnson Gary Earl JohnsonRepublicans not immune to the malady that hobbled Democrats What the numbers say about Trump's chances at reelection Presidential race tightens in Minnesota as Trump plows resources into state MORE in 2016.

Former GOP Rep. Mark Sanford Mark SanfordOn The Money: Business world braces for blue sweep | Federal Reserve chief to outline plans for inflation, economy | Meadows 'not optimistic' about stalemate on coronavirus deal Trump critic Sanford forms anti-debt advocacy group Republicans officially renominate Trump for president MORE (S.C.) has also said he is weighing a possible challenge against Trump.