U.S. stocks erased earlier gains to close lower Wednesday after the Federal Reserve released the minutes from its March meeting.

The Dow Jones industrial average closed about 40 points lower, with Goldman Sachs contributing the most losses. The 30-stock index had traded nearly 200 points higher earlier in the session.

The S&P 500 dropped 0.3 percent, with financials lagging. The Nasdaq composite slipped 0.6 percent after hitting a new all-time high earlier in the session.

The Dow and S&P also posted their biggest one-day reversal since February 2016.

The minutes showed Fed officials want to start unwinding the central bank's massive $4.5 trillion balance sheet later this year.

"I think the biggest surprise is the amount of optionality they included," said Matt Toms, chief investment officer of fixed income at Voya Investment Management. "They said they could be a more active Fed, which means the short end of the curve could be biased higher."

"On the balance sheet, they're talking about addressing both the mortgage and Treasury components. The market had expected them to address the mortgage component," he said.

Dow intraday chart

Source: FactSet

Unwinding the balance sheet is significant both because of its sheer size and the impact it could have on markets, as Fed members including Chair Janet Yellen have indicated that the move itself would amount to a rate hike.

"At some point, they have to lower it, but I don't think we'll ever see it back below $1 trillion," said Myles Clouston, senior director at Nasdaq Advisory Services. He also noted that some Fed officials had hinted earlier at lowering the balance sheet, "so this shouldn't be a surprise for investors."

The minutes also showed the central bank was concerned the stock market may be overvalued.

Investors also digested remarks from House Speaker Paul Ryan, in which he said that tax reform will take longer to accomplish than repealing and replacing Obamacare would.

"Right now, you have two pieces of news that shouldn't really surprise us but bringing down the market," said Art Hogan, chief market strategist at Wunderlich Securities. "Literally every Fed speaker has been talking about lowering the balance sheet."

The possibility of tax reform has been one of the main drivers in the stock market's massive postelection rally, along with deregulation and infrastructure spending.

"On the tax reform side, ... it's going to be difficult, no question," said Eric Stein, co-director of global income at Eaton Vance. "But financial markets should be forward looking mechanisms, so if something takes longer to happen, it shouldn't mean you" completely change your investment strategy.