(Read More: How the Student Loan Crisis Drags Down Home Prices)



Their story is getting ever more common, as total student loan balances nearly tripled between 2004 and 2012, according to a new survey from the Federal Reserve Bank of New York. Now $1 trillion in collective student loan debt is directly affecting the housing recovery.

"Short term, you see a decrease in the number of first-time home buyers," said Brian Coester of Coester Valuation Management. "You're going to see somebody who would have been able to afford a more expensive house maybe go for the lower version or the downgraded version."

First-time home buyers usually make up over 40 percent of the home buying population, but their share has hovered at or below 30 percent during this recovery, according to the National Association of Realtors. The student debt burden has kept many potential buyers out of the market, either forced to rent or to move back in with their parents, like Sophia Chaale.

"Without the student loan debt, a year and a half, two years earlier would have been the time I could have afforded to buy a house, and probably something a little bit bigger," Chaale said.

(Read More: Surging Student-Loan Debt Is Crushing the System

Chaale is facing $60,000 in student loans from graduate and undergraduate schools. She is paying $320 a month on a 30-year loan. Only after living at home for two years was she able to apply for a mortgage and put a down payment on a home. She is scheduled to close at the end of April.

"I consider myself lucky that I had a place where I could save, but what about other people who aren't originally from this area, who have to pay an extra $1500 a month in rent, and that rent money is not going to savings. How are they going to be able to save up or even to make that transition from renting to owning, in addition to all the student loan debt?" Chaale wondered.