The Federal Reserve announced Wednesday it would start to taper its aggressive bond-buying program to $75 billion a month beginning in January, propelling the market to a record close.

The FOMC also announced it would lower its monthly long-term Treasury bond purchases to $40 billion and mortgage-backed securities to $35 billion a month, both reductions of $5 billion.

(Read more: Here's what changed in new Fed statement)

"I think it logically, this is what they had to do," said David Kelly, managing director at JPMorgan Funds. "If you look at what's happened this year, the unemployment rate has come down to 7 percent. We've got housing starts over a million units. We got the S&P 500 up 25 percent. In this economy, you have to pull back from the most extreme monetary policy in a century. So I think it's overdue. I'm glad to see it."

Nine of the voting members of the FOMC supported the decision to start tapering. Only Boston Fed President Eric Rosengren dissented, noting the elevated unemployment rate and inflation below the 2-percent target.

Bernanke said in his final news conference as Fed Chairman that the recovery remains "far from complete," but if jobs gains continue as expected, the bond purchases would likely continue to be cut at a "measured" pace throughout next year.

Bernanke said he consulted closely on the decision with incoming Fed chief Janet Yellen.

"She fully supports what we did today,'' Bernanke said.