Larry Downing / Reuters

Obama’s plan for boosting the economy may seem huge—he proposed $447 billion in tax breaks and spending in his speech on Thursday night—but according to some economists, none of it will work until the problem that caused the economic crisis to begin with is addressed: housing. “The recovery will never turn into an expansion that we are comfortable with until housing has turned,” says Mark Zandi, chief economist for Moody’s Analytics, told my colleague Steve Gandel for a story he and I wrote a few weeks ago.

Fixing housing is crucial, say Zandi and other economists ranging from Harvard’s Martin Feldstein on the right to Yale’s John Geanakoplos on the left, because of the massive debt burden that remains over much of the economy and the continuing declines in home prices. While Washington has been focused on the federal debt, the U.S.’s $9.12 trillion of consumer real estate debt–down 10% from its 2008 record but still about triple what it was in 1999–has been a real drag on recovery. Home prices have dropped a staggering 40% since their peak in 2007, a worse drop even than during the Great Depression.

Consumer real estate debt and depressed home values kill economic recovery by keeping homeowners from spending money: people with enormous household debt can’t move for jobs and spend a disproportionate amount of their money trying to keep up with home payments. And even those with manageable debt constrain spending when their house loses value: for every drop in their homes’ value people spend 6¢ less on other stuff, according to economist Jared Berstein.

Obama briefly mentioned one idea to lighten the debt load and turn the housing market last night: refinancing much of the country’s housing debt down to today’s historic low interest rates. But some Congressional Budget Office economists say the impact of a refinancing program would be modest. (That’s not the CBO’s official position and other economists, such as Romney adviser and former Bush CEA chair Glenn Hubbard, argue it could produce tens of billions of dollars in consumer spending.) The administration is also pursuing a plan to rent foreclosed homes held by the government-owned Fannie Mae and Freddie Mac, thereby reducing the number of houses on the market and (they hope) boosting home prices. Obama’s top economic adviser, Gene Sperling, told the AP earlier this month that more housing plans may be on the way.

They should be, and they should include measures to cut principal debt, not just interest rates, for those who are current on their mortgage. That, says Geanakoplos, would reduce the debt burden for some 1.2 million homeowners. Obama should speed foreclosures for those who are seriously delinquent—evidence suggests local economies rebound faster when failing mortgages are liquidated quickly. And they should fund demolition of vacated houses that are not selling, are not needed for the rental program, and that are clogging the market with unnecessary supply, thereby depressing overall prices.