Investor confidence soared after the 2016 election of Donald J. Trump, sending the U.S. stock market on a tear that would last a full year.

It also made housing more expensive. That is because when the stock market rallies, the bond market usually sells off, and bond yields rise. Mortgage rates loosely follow the yield on the 10-year Treasury.

The average rate on the popular 30-year fixed mortgage hovered around 3.5 percent in the fall of 2016 and then shot up to as high as 4.3 percent immediately after the election, according to Freddie Mac. It stayed above 4 percent for the first half of the year and is currently just below that now.

The jump in mortgage rates, however, didn't immediately dampen consumer or builder confidence, which are both key to the housing market.

Builders were euphoric after the election, anticipating big deregulation in their sector. That has yet to happen, and now builders are fighting the Trump administration over its proposed tax plan, which cuts popular deductions for homeownership.

Housing starts and building permits are currently higher than they were a year ago, but the gains have been slower than expected. Builders complain of high costs for land, labor and materials. The Trump administration's anti-immigration stance has only hurt the construction labor shortage, which is largely fueled by Mexican workers.

"We need a national immigration reform bill. We have people that are afraid of being deported so they've gone underground. It's just made housing expensive," said Patrick Hamill, CEO of Denver-based Oakwood homes.