Stanley Fischer. REUTERS/Yuri Gripas Stanley Fischer, the vice chairman of the Federal Reserve, on Friday delivered an unusually sharp warning to President Donald Trump and his plan to "do a number" on post-crisis reforms aimed at reining in Wall Street.

Fed officials usually go out of their way to not appear political, which makes the comments all the more startling. Fischer, a former Citigroup banker and respected policymaker who led the Bank of Israel for many years, appears truly concerned.

"We seem to have forgotten that we had a financial crisis, which was caused by behavior in the banking and other parts of the financial system, and it did enormous damage to this economy," Fischer told CNBC's Sara Eisen in the lobby of the International Monetary Fund, responding to a question about the potential rolling back of Dodd-Frank rules.

This happened just as the president was signing an executive order aimed at what he said was "reviewing" Dodd-Frank.

"Millions of people lost their jobs. Millions of people lost their houses," Fischer said. "This was not a small-time, regular recession. This was huge, and it affected the rest of the world, and it affected, to some extent, our standing in the world as well. We should not forget that.

"The strength of the financial system is absolutely essential to the ability of the economy to continue to grow at a reasonable rate, and taking actions which remove the changes that were made to strengthen the structure of the financial system is very dangerous."

Asked specifically about Trump's vow to "do a number" on Dodd-Frank, Fischer shot back: "I'm not sure precisely what the president said and what a 'number' is, but there are aspects of Dodd-Frank, which if they were taken away would have very serious potential consequences for the economy — not immediately but when times get tough."

What provisions is he most worried about? The ability of the Fed and other regulators to wind down large banks, many of which are still seen as too big to fail.

"I think it is very important that big banks be subject to the discipline of the possibility of going bankrupt. It is also very important that that discipline extends to not making those changes, the bankruptcy of a big bank, a huge shock and the source of crisis or damage to the overall economy," Fischer said. "So we need the resolution mechanisms that have been put in place which will allow the authorities and the markets to wind up a big bank."

Watch the exchange: