When the Federal Reserve reversed course and began lifting short-term interest rates in December 2015, some forecasts called for mortgage rates to spike and chill the housing market.

It didn’t happen. But that doesn’t mean borrowers won’t see rates ratchet up in the months and years ahead, said Michael Fratantoni, chief economist with the Mortgage Bankers Association.

“We will get higher mortgage rates, but not that much higher,” he told an audience gathered Tuesday morning at the group’s annual convention, which was held at the Colorado Convention Center in Denver.

Fratantoni said rates on 30-year mortgages, which averaged 3.6 percent in 2016, are running closer to 4 percent this year. His forecast calls for them to average 4.6 percent in 2018, 5 percent in 2019 and 5.3 percent in 2020.

His forecast calls for economic growth that can be described as modest but strong enough to push down the unemployment rate. That will help push up wages, allowing more millennials and renters to purchase a home.

But rising wages also will boost inflation and contribute to higher borrowing costs. Another contributor will be Federal Reserve sales of mortgage-backed securities. The Fed, which holds $1.8 trillion of those debt instruments, plans to reduce its holdings to $1 trillion over the next few years. As more supply enters the market, higher interest rates are needed to entice buyers.

“It will be a factor in pushing up interest rates,” Fratantoni said.

Rate increases will be gradual enough to avoid pricing first-time buyers out of the market. But they will also coincide with a surge in millennials looking to purchase a home.

That increased demand, however, will exacerbate an ongoing problem: the inability of homebuilders to supply enough single-family homes.

Lynn Fisher, vice president of research and economics at the MBA, said gains in home prices and incomes ran in tandem through the late 1990s. Easy credit contributed to a spike in home values last decade, triggering a market crash that severely suppressed new-home construction for five years.

The downturn so severely crippled the home construction industry that builders, despite strong demand, haven’t achieved historic averages for new permits. That lack of supply is pushing up home prices at two to three times the pace of gains in incomes, especially on the lower end of the market.

“Home prices can’t keep accelerating at this rate,” said Fisher. But she doesn’t predict home prices will fall as they did late last decade. Rather, the rate of increase will slow.

Despite higher rates, the MBA forecasts the volume of mortgages taken out to purchase homes will continue to rise in the years ahead. But the modest rate increases this year and last have put a huge dent in the refinance market, reducing a major source of lending activity.

Mortgage refinances, which approached $1 trillion in 2016, are on track to reach only $600 billion this year and will fall to $430 billion by 2018, Fratantoni said.