Washington should stay away from touching the mortgage interest tax deduction, warns the U.S. housing industry.

Lately, housing is on the mend and one of the few bright spots in a lumbering economic recovery. Taking away a key tax break could throw a wrench into home buying plans and hurt a long-sputtering recovery.

Lawmakers in both parties are on the lookout for tax revenue as a way to avert the fiscal cliff.

But the housing industry is preparing to fight against any move to get rid of the mortgage interest tax break.

"[Getting rid of it] would throw the housing sector into turmoil ... and chill the market just as it is trying to recover," said Jerry Howard, CEO of the National Association of Home Builders.

Powerful housing lobbying groups are taking their fight to the grass roots, armed with granular data on the benefits of the homeowner tax break in every congressional district.

Lobbyists from the industry have spent a combined $30 million this year, up from $27 million last year, according to Center for Responsive Politics figures. The bulk of that came from the powerful group, the National Association of Realtors, which spent a record $25 million on lobbying this year, more than any other year, federal records show.

They're ensuring that leaders don't do anything "penny-wise and pound foolish," said David Stevens, CEO of the Mortgage Bankers Association.

Related: Mortgage rates fall to record lows again

This isn't the first time Washington has taken a critical look at the mortgage interest tax deduction.

It is one of the oldest tax breaks and designed to encourage home ownership, by lowering the tax bill for homeowners.

It tends to benefit upper middle class families the most, according to the Tax Policy Center. For those earning more than $250,000 a year, the annual tax savings run about $5,460. For those with annual incomes of less than $40,000 a year, the average savings is just $91, according to the center.

The deduction is the third largest tax expenditure on the federal budget, according to the Congressional Research Service. The amount of revenue the government would forgo from those claiming mortgage interest deductions is estimated to reach $100 billion by 2014.

President Obama has proposed in his budget a cap on itemized deductions to 28% of gross income from 35% for high-income Americans. The cap would apply to many popular deductions such as mortgage interest and charitable donations.

But Obama's proposals have gotten nowhere, thanks to lobbying from home builders, the National Association of Realtors and the Mortgage Bankers Association.

But this time, lobbyists are worried. That's because for the first time in years, House Republicans say they are open to scrubbing any tax breaks from the books as part of shrinking federal deficits.

Stevens of the Mortgage Bankers Association said the economy "could actually move backwards" if the deduction is taken away, he warned because it has a significant impact on middle class Americans' cash flow.

Related: Most affordable cities for homebuying

The National Association of Realtors, which has spent the most on lobbying this year, declined to share its plans on defending the deduction. But earlier this month, president Gary Thomas touted that the NAR had "secured 183 bipartisan cosponsors," this year to support a House resolution that would protect the current tax deduction for mortgage interest.

"We will continue to work with members of Congress on the consumer's behalf on this issue," Thomas said in a statement.