If Gary Cohn ever wants to head the Federal Reserve, then he's likely to stay right where he is in the Trump administration.

As speculation swirls over whether the president's top economic advisor might be the latest White House official to bolt, the Fed factor could prevail over all others. President Donald Trump himself has signaled that Cohn is a top contender for the seat should current Chair Janet Yellen be replaced.

However, Cohn is thought to be deeply disappointed in Trump's behavior since the racial violence in Charlottesville, Virginia, a week a week ago. The question is whether he is upset enough to leave his White House post and, in doing so, squelch virtually any chance he'd have at the Fed position.

"It's a difficult question," said Christopher Whalen, head of Whalen Global Advisors and himself once rumored for a Fed governorship appointment.

As Whalen sees it, the credibility of the Trump administration, particularly at such a tumultuous time, depends on having stable hands around like Cohn, Treasury Secretary Steven Mnuchin and chief of staff John Kelly.

"He and Kelly and Mnuchin are all lending their credibility to this presidency. At a personal level, they're going to have to make a decision," Whalen said. "On the one hand, you want to serve the people of the United States. At the same time, you are basically lending your support to what he does by serving him."