NEW YORK (Fortune) -- Oil prices are falling sharply, and that's good news. But not nearly as good as you might think.

No doubt the drop, down to $120 by mid-day Monday, gives strapped consumers relief at the gas pump. Prices have dropped below $4 a gallon and could be headed toward $3.50, going by trading in wholesale futures markets. Any decline will be welcomed by Americans struggling under the burden of falling house prices, rising layoffs and stagnant wages.

But falling oil prices also suggest that the recession the U.S. has so far avoided is well on its way, as consumers pull back from the spending spree that drove economic growth earlier this decade. A weakening economy will mean more layoffs, further pressuring already reduced spending.

"There is no doubt that with gasoline prices dipping below $3.90 a gallon we have a bit of a reprieve on the energy front," Merrill Lynch economist David Rosenberg wrote in a report Monday, "but the reality is that this is a chicken and egg game because the decline is reflecting the consumer recession."

Energy use down

Perhaps the biggest factor behind the recent 18% drop in the price of a barrel of crude is sinking North American demand. Federal Highway Administration data show the number of miles driven in the U.S. dropped from year-ago levels for the seventh straight month in May.

May's decline was the third-largest monthly drop on record since 1942, says Stephen Schork, editor of the Schork Report energy and shipping newsletter in Villanova, Pa.

Americans are driving 4% less now than they were a year ago, Rosenberg writes, while energy use in inflation-adjusted terms has dropped 2% - an event he calls "extremely rare."

The pullback comes after the recent crude-price surge - the cost of a barrel doubled between Labor Day of 2007 and July 11 - seriously damaged the industrial economy, which despite its long decline remains a crucial source of better-paying jobs.

General Motors (GM, Fortune 500) on Friday posted a $15.5 billion second-quarter loss, as sales plunged 18% from a year ago. The company and rival Ford (F, Fortune 500) have slashed truck production, laid off thousands of workers and refocused on smaller cars as buyers flee the light trucks that had made the companies so much money.

Americans' decision to drive less comes at a time of rising stress. The economy has been hemorrhaging jobs and real wages, adjusted for inflation, have been flat to lower for a decade. Americans have enjoyed a rising standard of living in the meantime by borrowing - but with banks choking on subprime mortgages gone bad, the loan window is closing. Rosenberg calls a recent rise in the savings rate "a vivid sign that frugality is now replacing frivolity."

Meanwhile, the weak economy is spurring more companies to cut back. Outplacement firm Challenger Gray & Christmas said Monday that layoff announcements jumped 26% from a month ago in July. The unemployment rate recently hit a four-year high at 5.7%.

How low can it go?

One unhappy fact is that a drop in the price of oil won't bring back many of the jobs lost over the past year to the energy-cost surge. Even were gas to fall to $3 a gallon - a move that is by no means assured - no one is going to beat a path to the dealership to buy pick-ups and SUVs that are now, in many cases, being phased out. GM recently announced plans to shut four SUV plants.

On a happier note, there is hope that the decline in oil prices has just begun. While Schork says it's anyone's guess where crude will trade - "By the end of the third quarter, there's a good chance oil could be below $100 a barrel, and a good chance it could be above $150," he says - others see a chance that the commodity, having enjoyed a head-spinning runup, could also drop more than anyone expects. Economist Jim Griffin notes at the ING Investment Weekly that crude's rally earlier this year became "nearly parabolic" - a sign that the decline could be steep.

Now a return to double-digit oil may not rescue the Hummer. But as the government's fiscal stimulus program did earlier this year, it could give consumers a little more change in their pockets, either to spend, salt away - or pay down their debts.