MANCHESTER, N.H. -- Former House Speaker Newt Gingrich sharply criticized comments made by Mitt Romney during Sunday morning's debate, when he suggested that Americans who need to pay off their mortgage should perhaps not run for public office.

Romney said his father once told him, "'Mitt, never get involved in politics if you have to win election to pay a mortgage.' If you find yourself in a position when you can serve, you ought to have a responsibility to do so if you think you can make a difference. He said also don't get involved in politics when your kids are still young because it may turn their heads."

Romney's advice was meant to warn against the idea of a career politician who is simply interested in staying in office and profiting from public service. But arguably, there are less-affluent citizens who may want to answer the call of service and may nevertheless need a paycheck from public office in order to pay the bills.

At a Hispanic town hall meeting at the Don Quijote restaurant in Manchester on Sunday, Gingrich sharply criticized Romney's remarks.

"This morning, when Gov. Romney made the comment that you shouldn't run in order to pay your mortgage -- I thought that was very much the opposite of the American tradition historically," he said. "We want everyday, normal people, to be able to run for office. Not just millionaires."

Gingrich said he was raised as an "Army brat" whose family didn't have much money, and that he was able to run for office largely with the help of the Republican Women's Federation.

"It was harder than it should have been," he said. "But today it's even harder than it was back then. ... So I think it's really important we get back to making it possible for everyday middle class candidates to go out and run for office."

The intersection of money and politics played an outsize role at Gingrich's town hall.

When one woman asked him about how he would take the influence of money out of politics, Gingrich replied, "I think it is a practical reality that people who want to influence a government of this side are going to figure out a way to do it. ... The answer, I think, is a very simple election law that says, anyone can give any amount of personal after-tax income to the candidate, as long as they report it that night on the Internet."

There were protesters outside the restaurant calling to get rid of corporate money in politics. Some of them were banging on drums and shouting through megaphones throughout the event, and inside, Gingrich had to speak over the din.

At one point, a man named Paul who identified himself as an Occupy Wall Street protester interrupted Gingrich and asked whether he would decline corporate contributions. When Gingrich responded, the man repeatedly tried to debate him until the former House Speaker requested that he give other people a chance to speak, and Gingrich's staff came over and asked him to be quiet.

In 2002, Gingrich actually voiced concerns similar to those expressed by the protesters today. During a debate with political activist and former presidential candidate Ralph Nader, Gingrich said, "I'm very happy to get corporations out of politics. I think it'd be better for America if you had no union and corporate donations, but individuals could donate of their own after-tax income."

When asked by The Huffington Post on Sunday about his earlier comments, Gingrich replied, "I think it'd be better to have individuals give unlimited personal money, and then you wouldn't need corporations and unions."