"Consumers who are pessimistic about current housing market conditions are more likely to cite unfavorable economic conditions than the prior month," said Doug Duncan, Fannie Mae's chief economist. "Job confidence remains high but still well shy of its July reading."

Consumer sentiment on housing fell in September from its August high, according to a monthly survey from Fannie Mae. While more respondents think now is both a good time to buy and sell a home, there was a much larger drop in the share of those who said they were not concerned about losing their jobs. It was the second straight month that the component of the survey fell.

Lower mortgage rates are making buying a home slightly more affordable, but financial concerns are outweighing that benefit and lowering overall confidence in housing.

The share of those saying their household income is significantly higher than it was a year ago was unchanged at just 21%. Although there was improvement in both buying and selling sentiment, far more consumers think now is a good time to buy rather than sell.

The survey comes as mortgage rates sit at the lowest level in over a month and are significantly lower than they were a year ago. While rates did jump in September, they were back down by the end of the month.

With the average rate on the 30-year fixed mortgage around 3.64%, only about 21% of the national median income is required to make the monthly principal and interest payment on the average-priced home. This is the second-lowest payment to income ratio in 20 months, according to a new report from Black Knight Inc.

The average monthly payment on the average-priced home is now 10% lower than it was last November, when mortgage rates peaked around 5%. That even includes a 4% home price increase since then.

"Back in November 2018, we were reporting on home affordability hitting a nine-year low," said Black Knight Data & Analytics President Ben Graboske. "Interest rates were nearing 5%, pushing the share of national median income required to make the principal and interest (P&I) payments on the purchase of the average-priced home to 23.7%. While still below long-term averages, that made housing the least affordable it had been since 2009, spurring a noticeable and extended slowdown in home price growth."

The drop in mortgage rates has pushed the average monthly payment down by about $124 from November of last year. That in turn boosts buying power by $46,000. In other words, lower rates today mean a buyer can purchase a home that costs $46,000 more and pay the same monthly payment as they would have last November on the cheaper home.

Home prices are still rising, but the growth eased throughout much of this year and then flatlined in August.

"It remains to be seen if this is merely a lull in what could be a reheating housing market, or a sign that low interest rates and stronger affordability may not be enough to muster another meaningful rise in home price growth across the U.S.," noted Graboske.

The key factor fueling prices continues to be low supply, and it has not increased meaningfully in a few years now. Low mortgage rates could help, giving homeowners who already have low rates more incentive to move and not lose that rate. As rates rise, more owners tend to stay in place, unwilling to pay higher interest rates for the same debt.

Of course all real estate is local, and affordability varies from market to market. California continues to be the worst, with 7 of the 10 least affordable housing markets in the nation. In Los Angeles, it currently takes 43% of the median household income to be able to purchase the average-priced home. That's an improvement from the 48% required at the end of last year, but it still ranks as the least affordable market in the nation.