Summers had particularly hash words for Wal-Mart CEO Doug McMillon, who on Tuesday issued a statement criticizing Trump but saying he's staying on the president's economic council.

There's always room for debate on policy positions, Summers said. "But this is traducing basic American traditions of justice, inclusion, and fairness."

The former Clinton Treasury secretary and ex-Obama economic aide said on " Squawk on the Street " the controversy over President Donald Trump 's comments since Saturday's deadly white nationalist rally in Charlottesville, Virginia, is "unlike anything we've seen before."

Any CEO who stays on any of the White House business advisory councils will face backlash because they'll be on the wrong side of history, Larry Summers told CNBC on Wednesday.

"[McMillon] is not fit to be the CEO of Wal-Mart if his judgment is so bad as to suppose that being a member of this council is giving him some kind of effectiveness," Summers argued. "What he may be — and what a number of CEOs are — is scared; scared that if they leave, the president using the tools of government will retaliate."

If Trump has created a culture of fear among American CEOs, "that is a particularly damning indictment far beyond any in Charlottesville of his presidency," Summers said.

Wal-Mart declined to comment on Summers' remarks.

On Monday, Merck CEO Ken Frazier resigned from the White House manufacturing council, igniting the debate about whether corporate leaders should leave Trump's advisory panels over the president's response to Charlottesville.

Less than 24 hours later, Intel CEO Brian Krzanich and Under Armour CEO Kevin Plank followed suit.

On Tuesday, AFL-CIO President Richard Trumka, union colleague Thea Lee, and the president of the Alliance for American Manufacturing, Scott Paul, resigned as well.

"I cannot understand why others have not followed Ken Frazier out the door. I cannot understand why they have not endorsed his courageous action," said Summers. "This is not a happy day for American business."

Summers spoke before a member of Trump's Strategic and Policy Forum told CNBC the panel has agreed to disband. Also Wednesday, Campbell Soup chief Denise Morrison and 3M CEO Inge Thulin resigned from Trump's manufacturing council.

TWEET 3M

In addition to calling out corporate America, Summers also said Wednesday he's surprised no officials of Trump's administration had resigned.

"There's a tradition in America of people of principle when they are sufficiently offended and disagree with their president on matters fundamental resign," he said.

"I have been disappointed that there have been no resignations on principle by the political appointees of this administration."