Many people living in rentals were once owners; they lost their homes to foreclosure and now have such damaged credit reports that they find it nearly impossible to qualify for a mortgage. Others are trapped because lenders have significantly tightened credit standards after the abuses of the boom era.

And while the federal government has created programs to encourage lenders to offer mortgages requiring only a small down payment, the efforts are so nascent that officials won’t say how many people have taken advantage of them.

Apart from the hangover from the housing collapse and the worst economic downturn since the 1930s, the nation’s changing demographics are also causing a major shift in housing trends. For instance, a majority of new households expected to be formed in coming years will consist of people with a minority background. Historically such Americans have had lower incomes and fewer assets and were less able to buy homes, according to the Urban Institute.

At the same time, millions of young adults who normally would be first-time home buyers are still struggling to find decent jobs; many are also putting off marriage and having children, a trigger for home buying. They are also more likely than previous generations to be saddled with heavy student loan payments that hurt their ability to save for a down payment.

But it is not just younger people who are having trouble owning a home. According to the Joint Center’s report, that rate dropped the fastest for people in their late 30s to early 50s. These people were in their prime home-buying years right before the recession; when housing prices plummeted, they were left with little or no equity.