AP Oliver Wendell Holmes, the U.S. Supreme Court justice pictured here in 1931, was known to "enjoy" his taxes. "With them I buy civilization," he said.

Mitt Romney has proclaimed, “I pay all the taxes that are legally required, not a dollar more.” While most Americans may not agree with his tax rate (14%), few would disagree with his sentiment. Almost no one willingly pays more taxes than required. Yet there was a time when elites willingly acknowledged that they should pay far higher rates than others. In fact, when the income tax was first regularly put into place in 1913, the well-off were the only ones required to pay it.

Before that time, our government relied upon a system of tariffs for revenue. But as the nation grew in population and economic prowess, it required public works like more efficient transportation to grease the wheels of capital, public goods like schools to create more skilled workers, and public protections to help those who the market failed. Our leaders — many of whom came from the wealthy elite — turned to the idea of an income tax. In 1909, Republican president Teddy Roosevelt argued in favor of income and inheritance taxes, as they would promote, “equality of opportunity.” The programs required a constitutional amendment, and by 1913, 88% of states agreed that it was time to tax the income of its citizens. But not all its citizens — instead the income tax burden fell solely on couples who made over $4,000 (in today’s terms, around $88,000). If you made less, you paid nothing. And the more you made, the more you paid.

(MORE: One Nation, on Welfare: Living Your Life on the Dole)

For the next 60 years Americans lived under a progressive tax structure. And while elites were not overjoyed to pay higher taxes than other Americans (and some sought ways to avoid them), most understood their tax burden as their civic duty. Franklin Delano Roosevelt argued that, “Taxes shall be levied according to ability to pay. That is the only American principle.” Supreme Court Justice (and Boston Brahmin) Oliver Wendell Holmes was known to “enjoy” his taxes. According to Felix Frankfurter’s book Mr. Justice Holmes and the Supreme Court, Justice Holmes told a young law clerk who complained about paying them, “I like to pay taxes. With them I buy civilization.” When John D. Rockefeller Sr. died in 1937, the estate tax was nearly 70%, yet complaints from his family would not be publicly heard. Two years earlier his son earned more than $5 million; this gave him the distinction of being the only person in America’s highest tax bracket (at a rate of 63%). No editorials were written by John Jr. to suggest class warfare, or that the rich were being unfairly singled out.

Yet in the 1970s, this attitude began to change. Elites began to use their increasing political power to lower their taxes, an effort that came to fruition under Ronald Reagan. Today our elites employ what political scientist Jeffrey Winters calls “the income defense industry” to greatly reduce their taxes. These lawyers, lobbyists, and tax consultants have worked to protect the wealthy with special laws, legally move money to special places like the Cayman Islands and Switzerland, and undermine progressive taxation.

(MORE: The Phony Battle Over the Self-Made Man)

The results have been successful. As economists Thomas Piketty and Emmanuel Saez have shown, between 1970 and 2004 the rich cut their overall effective tax rate by more than half — from 75% to 35% — while the American average has remained at 23% (this includes individual, corporate, payroll and estate taxes). But at what cost? Inequality has increased, mobility has declined, and the American Dream is dying. Not every wealthy American believes they are being robbed — both Warren Buffet and Bill Gates have taken the position that loopholes for the wealthy should be closed, taxes should be raised, and billionaires should pledge to give away at least half their fortunes. (When Warren Buffett was married in 1952 he had a discussion with his new bride about the fact that they were going to be rich. “We agreed with Andrew Carnegie, who said that huge fortunes that flow in large part from society should in large part be returned to society.”) But the rich who lobby against progressive taxation have replaced the civic principle of taxing on the basis of the ability to pay with the individual principle that nothing should get in their way of being as rich as possible.

MORE: Rich People Problems? The Summer of Elite Disconent