A jump in interest rates has had a big impact on the housing market. That's a warning sign for a Federal Reserve seemingly bound and determined to withdraw stimulus from a still-shaky economy.

The mortgage market cratered in the fourth quarter of 2013, Wells Fargo and JPMorgan Chase reported on Tuesday, as higher interest rates all but murdered demand for mortgage refinancing.

The nation's biggest mortgage lender, Wells Fargo, said its mortgage volume tumbled to $50 billion in the quarter, down 60 percent from $125 billion a year ago. The second-biggest lender, JPMorgan Chase, said its mortgage originations, that includes new home purchases and refinancings, fell 54 percent to $23.3 billion from $51.2 billion a year ago. (Story continues after chart of ugliness.)

These two banks are key indicators for the state of the housing market. Wells Fargo issued nearly 23 percent of all U.S. home loans last year, to JPMorgan Chase's 11 percent, according to industry tracker Inside Mortgage Finance.

And the story they're telling is stark: Higher rates have hurt demand. The average interest rate for a 30-year fixed-rate mortgage has jumped to 4.5 percent from a record low of 3.3 percent in early 2013, according to government-backed mortgage giant Freddie Mac.

This has happened mostly because of the Fed. Central bankers last year repeatedly declared a desire to pare back, or taper, their program to buy $85 billion in mortgage-backed securities and Treasury bonds each month to keep interest rates low. They made good on that promise last month, trimming $10 billion from the program. Despite a still-anemic job market, most economists think the Fed will trim another $10 billion at its next policy meeting, later this month.

Fed Chairman Ben Bernanke and others argued they weren't kicking the props out from under the bond market, but that's sort of what happened: Bond prices fell, and interest rates jumped.