William Dudley, former New York Fed president, has a message for Wall Street: Stop blaming the central bank when the stock market declines. Dudley, who retired in June 2018, said Thursday on CNBC that the Federal Reserve's actions have become a "convenient whipping boy."

Many traders blamed the late 2018 market tailspin on the Fed's balance-sheet runoff. The program, which allows bonds in the Fed's portfolio to mature without replacing them, effectively reduces the central bank's balance sheet. The balance sheet had swelled to more than $4.5 trillion in early 2015 in the wake of several rounds of quantitative easing, or bond-buying, aimed at boosting the economy after the 2008 financial crisis.

The runoff is expected to end later this year, with the balance sheet at around $3.7 trillion. Reducing the balance sheet, like hiking interest rates, acts as a tightening measure on the economy.

"Look what happened. The balance sheet is still running off, and the stock market has recovered in the first quarter," Dudley told CNBC's Steve Liesman, in an interview from the annual conference of the National Association of Business Economics in Washington, D.C.

"It was a convenient whipping boy; the Fed's seeming inflexibility in the space of all these market developments for a while was a convenient whipping boy. The markets occasionally go down for a whole host of reasons," Dudley said.

The "seeming inflexibility" refers to Fed Chairman Jerome Powell's early October remarks that indicated a more aggressive path for rates in 2019 and December comments that the balance-sheet runoff was on autopilot. After hiking rates four times in 2018 and projecting two hikes for this year, the Fed held steady at its January policy meeting, and the Powell Fed said it would be "patient" in determining further moves.

The Fed has always been data dependent, said Dudley, who argued that five things happened to change monetary policymakers' minds between October and January. "Number one, financial conditions tightened in the fourth quarter, quite significantly. Number two, despite strong payroll growth, the unemployment rate stopped declining. Number three, the tightness in the labor market didn't lead to much wage acceleration. And number four, inflation was a little bit on the soft side. And number five, there were questions about global growth, China and Europe."

None of those factors alone would have changed the Fed's stance, but all of them together pointed to more downside risk in the economy, Dudley said. "With inflation not being a problem, I think they decided to, why not just wait for more information. It doesn't mean they're done. They just want to see more information."

Despite the concerns about the global economic slowdown spilling over into the United States, Dudley said he does not see a recession in America anytime soon. Prior to joining the New York Fed, Dudley served at Goldman Sachs as chief U.S. economist and a managing director.