NEW YORK (Fortune) -- House prices are taking a long ride in the wayback machine. Unfortunately, Americans' housing-related debt isn't going anywhere fast.

Prices in big U.S. cities posted their biggest-ever decline in the first quarter, according to the most recent S&P/Case-Shiller National Home Price index. After nearly three years of declines, house prices nationwide are back at 2002 levels -- and still falling.

Yet as bad as that is for overburdened homeowners and their bankers, the mighty mountain of mortgage debt Americans have taken on is an even bigger concern - especially for those who believe an economic recovery is in sight.

Even though the amount of home mortgage debt outstanding declined in 2008 for the first time since the Federal Reserve started keeping track in 1945, mortgage debt levels remain distressingly high.

Home mortgage debt outstanding was 73% of gross domestic product last year, according to government data. That's the third-highest reading on record, after the 75%-plus bubble years of 2006 and 2007.

Getting that ratio down to a more manageable number will mean more lean years ahead, as Americans further cut spending to rebuild their savings and banks struggle to boost their capital amid heavy loan losses.

How long this process might take is a key question for those trying to gauge the prospects for an economic recovery.

To get the mortgage debt-to-GDP ratio down to a more normal level such as the 46% average of the 1990s, Americans would have to cut their mortgage debt to $6.6 trillion from $10.5 trillion at the end of 2008. The last time the national mortgage debt count was below $7 trillion was 2003, according to Federal Reserve data.

We might call this mortgage overhang the $4 trillion elephant in the room for policymakers, who have spent the past year injecting liquidity into the economy - a course of action that will do little to solve the problem of too much debt.

Of course, these figures reflect only back-of-the-envelope estimates. Depending on the level of interest rates and how successful officials are in restoring the vigor of the lending markets, mortgage debt may or may not need to drop that far to relieve some of the stress on consumers.

Still, there is little doubt that above-average debt levels will impede the sort of consumer-driven economic rebound that has taken place after the last few recessions.

"I don't think that there is any magic to the '90s debt levels," said Dean Baker, an economist at the Center for Economic and Policy Research in Washington. "The point is that with higher debt levels, people will be consuming less."

Borrowers who are overstretched on their mortgages are less able to spend money on other goods and services, and more apt to fall behind on payments. That could mean more painful writedowns ahead for already troubled banks.

The scale of the mortgage overhang may help explain why, even after banks such as JPMorgan Chase (JPM, Fortune 500) and Citigroup (C, Fortune 500) received generally upbeat stress test results, some prominent skeptics of the housing bubble are warning that the financial system's problems aren't over.

"There is still a very large unfunded capital requirement in the commercial banking system in the United States and that's got to be funded," former Fed chief Alan Greenspan said last week in an interview with Bloomberg. He added that "until the price of homes flattens out we still have a very serious potential mortgage crisis."