"Starting in our third quarter we really articulated the fact that over the past few years we've seen sales prices moving up rather rapidly. Now, when you layer on top of that the very quick moves in interest rates that we saw from the Fed, that translated into the mortgage rates, it really created an element of sticker shock," said Stuart Miller, chairman of Miami-based Lennar in an interview on CNBC's The Exchange.

New home sales fell hardest in California, where prices are highest, according to JBRC. Sales dropped 40 percent annually in northern California and 49 percent in southern California. Sales were just 5 percent lower in the Midwest, where homes are far less expensive.

The California numbers are in line with the latest quarterly earnings figures from Toll Brothers, the nation's largest luxury homebuilder. It reported a 39 percent drop in new orders from California.

"In November, we saw the market soften further, which we attribute to the cumulative impact of rising interest rates and the effect on buyer sentiment of well-publicized reports of a housing slowdown," said Toll's CEO Douglas Yearley in the release. "We saw similar consumer behavior beginning in late 2013, when a rapid rise in interest rates temporarily tempered buyer demand before the market regained momentum."

While buyer traffic was lower in November and December compared with previous years, it has remained steady since August, according to JBRC's survey. Demand for housing is still very high. Mortgage rates did fall back in December, and builders like Lennar reported seeing a correlating jump in buyer traffic through model homes, though not contract signings.

"We saw traffic seasonally adjusted flat from November to December, so maybe Lennar's were up and everybody else's were down a little bit," said Burns. "I'm expecting lower rates to help, but it's really not causing a significant recovery."

Just under one quarter of builders surveyed by JBRC said they reduced prices, and that may have helped sales in frothy markets such as Seattle and Portland, OR. Cancellations, however, rose compared to a year ago, with entry-level buyers pulling the plug most.

Their cancellation rate was 18 percent, compared with move-up buyers who cancelled 14 percent of deals and luxury buyers who pulled out of 11 percent of deals. The rate for luxury buyers was higher than a year ago, which may be due to heavy turbulence in the stock market at the end of the year.

John Schoen contributed the graphics for this article.