"What has been the story for the last six months is that at least he has decent people around him ... and that would change instantly" if Cohn or Mnuchin were to leave, said Art Cashin, UBS director of floor operations at the New York Stock Exchange. Cashin said the selling accelerated Thursday when Trump denounced the removal of Confederate statues in a tweet, during morning trading. The market has been particularly uneasy about Cohn, who was reportedly appalled after Trump turned an infrastructure announcement on Tuesday into a tirade about Charlottesville. The looks on the faces of the advisors surrounding him ranged from disgust to disbelief. "Cohn's probably the front-runner to replace Janet Yellen, so I can't see him voluntarily giving up an opportunity like that. I don't see him getting fired. I think he's held in high regard. It was clear to me and I think a lot of people on Tuesday that he was distressed by Trump's doubling down on the Charlottesville stuff, but I don think it's sufficient for him to quit," said Greg Valliere, chief global strategist at Horizon Investment. "I think like everyone at the White House, he's distressed about what Trump has done." Yellen's term ends in February, and Trump has said that Cohn and Yellen are in the running. The latest White House drama has investors wondering what impact an exodus of administration officials would have on a stock market that seems to have sailed past the president's prior blunders, like the firing of FBI Director James Comey and the investigation into his campaign's dealings with Russia. Strategists say the biggest market reaction, if any, would come from the loss of Cohn or Mnuchin, both of whom are perceived as key drivers of tax reform. "I think [Cohn's] probably worth 200 [Dow] points only because if he goes, they'll assume others are going to go too," said Cashin. "He's got to be conflicted. The CEOs have to worry about a different thing. They have to worry about losing business or being boycotted. If you're an individual alone, and you're on the short list to be the Fed chairman, you think you can stay there and you think you can do good for the country." Yale University management expert Jeffrey Sonnenfeld said Thursday on CNBC that Cohn is so key, his departure could "crash the markets." "I don't want to be an alarmist, but there is a lot of faith that he is going to help carry through the tax reform that people are looking for," he said on "Squawk Box." "I think if he steps away, it would crash the markets," said Sonnenfeld, senior associate dean for executive programs at Yale's School of Management.

Sam Stovall, chief market strategist at CFRA, said he doubts the market reaction to a resignation would be consistent with a crash. "If you have the departure of a respected and potentially influential person, that could have a negative near-term effect on the market, but then the market just reverts to its old tendencies, which is being opportunistic. And that leads to the bigger question of will his departure lead to a recession. If the answer is no, you buy the dip. I think we would end up with some softness in the market, and then move forward," he said. Jeannette Lowe, a policy analyst at Strategas Research, said either Cohn or Mnuchin would be an important loss. She pointed to the market sell-off around the Senate's efforts to vote on health-care reform. "You could see a similar move down. We still could bounce back, but that's saying you've lost someone steering the ship and that could be a little bit more of a market mover," said Lowe.

Director of the National Economic Council Gary Cohn and Treasury Secretary Steve Mnuchin look on as US President Donald Trump delivers remarks following a meeting on infrastructure at Trump Tower, August 15, 2017 in New York City. Getty Images

Lowe said Mnuchin appears to be happy in his role and is given free reign, so he does not seem ready to exit anytime soon. Both Cohn and Mnuchin are set on getting tax legislation passed, and if either left, it would be more likely to be after a tax package, she said. "They're the really moderating forces. Investors would definitely see that as a negative and discount for tax reform getting done," if they were to leave, Lowe said. Valliere said if anyone left the administration, it's more likely it would be White House chief of staff John Kelly. Valliere said he could be gone by Thanksgiving. On Tuesday, Kelly looked particularly distressed during Trump's briefing. "If you saw that snapshot..His body language. I think he was aghast. Absolutely, they had not planned to go there and Trump violated what they agreed on. Kelly's had no impact on Trump," said Valliere. He said Trump continues to tweet whatever he wants.

White House Chief of Staff Gen. John Kelly looks on as US President Donald Trump speaks following a meeting on infrastructure at Trump Tower, August 15, 2017 in New York City. Getty Images

"The market continues to do well despite all of this stuff. Even with more and more defections among Republicans, I think they're just going to go off and do their own thing. We may get something done in spite of Trump rather than because of him," said Valliere. White House advisor Steven Bannon is unlikely to leave, but Trump could throw him overboard if he needs a scapegoat, Valliere said. Bannon gave an interview in which he said the U.S. has no military options with North Korea, and that he would make personnel changes in the State Department. "He's available to be thrown under the bus. He said a lot of things I think the president would have issues with. ... He did say a lot of provocative things. I would say he's on thin ice. I think Trump is holding him in reserve. He may need a scapegoat down the road," said Valliere. "I think Bannon probably stays." Valliere said the market would not necessarily see Bannon's departure as a positive, even if he does represent the far-right wing of Trump's base. "Market reaction would depend on his departure, but it could reinforce the concerns the White House is in total disarray, and I don't think that's a good story. I think if Bannon were to leave there would be instant speculation about what damage he could do to Trump with an interview or two ... and I think Trump is aware of that." Lowe said she believes Secretary of State Rex Tillerson is more likely to leave than any of the others. "I think there's been a lot of talk about it but we haven't heard anyone actually say we feel someone is going to walk away. I think the biggest concern is Tillerson," Lowe said. Beyond stocks, the departure of Cohn or Mnuchin could have big implications in the bond and currency markets. BMO fixed income strategist Aaron Kohli said if Mnuchin were to leave it would raise concerns about the whole budget process and debt ceiling, in addition to tax reform. The budget needs to be approved by the end of September, which is also when Mnuchin warns the U.S. will hit the debt ceiling. "Mnuchin is seen as the driver of the tax proposal. He is seen as the voice of reason," Kohli said. "The fact the debt ceiling is coincident with the budget makes each one more difficult." But it's less clear to Kohli which way the market would move if Cohn were to depart in the near future. "If he's out, the question would be who, if not Yellen?" for Fed chairman. Strategists say that they still expect Republicans to push through tax reform, and that the market is currently not pricing much hope for it.

WATCH: Cohn resignation would 'crash the markets,' says Jeff Sonnenfeld