Sen. Lindsey Graham (R-S.C.) broke with anti-tax crusader Grover Norquist on Tuesday, telling ABC's Jonathan Karl that he supported eliminating tax deductions in order to help get the country back on solid fiscal footing.

"We are so far in debt that if you don't give up some ideological ground, the country sinks," Graham said.

Graham is one of over a thousand Republicans nationwide who have signed Norquist's anti-tax pledge to oppose and vote against any effort to increase taxes.

The pledge also opposes raising revenue by eliminating tax deductions and credits. Graham voiced his disagreement with that component, saying "when you eliminate a deduction, it's OK with me to use some of that money to get us out of debt."

He praised Norquist for "doing a great service" but said that due to the country's poor fiscal climate, the Republican party's position must evolve.

"When you talk about eliminating deductions and tax credits for the few, at the expense of the many, I think over time the Republican party's position is going to shift. It needs to, quite frankly, because we are $16 trillion in debt," he said.

"I'm willing to move my party, or try to, on the tax issue. I need someone on the Democratic side being willing to move their party on structural changes to entitlements."

Mitt Romney, the GOP's presumptive presidential candidate, signed the anti-tax pledge before his 2008 run for office.

Another GOP heavyweight, former Florida Gov. Jeb Bush, criticized Norquist's pledge recently, saying, "I don’t believe you outsource your principles and convictions to people.” He urged Romney to accept some tax increases in order to reduce the government’s budget deficit.

Asked whether Romney agreed with him, Graham said he wasn't sure. "Someone needs to ask him," he quipped.

Earlier on HuffPost:

House Republicans argue the rich already pay their fair share in taxes:

PHOTO GALLERY Republicans: Wealthy Already Pay Enough In Taxes