I have written in the past about the excessive compensation of the CEOs of the largest banks. However, it has become clear to me that the issue of inflated pay packages does not lie solely or even primarily with the large banks. Rather, the problem is endemic to corporate America. Indeed, perusing a list of company executives ranked by compensation, one has to get past 50 firms before one finds the CEO of a bank holding company. Nonbank executive remuneration has grown faster than that for banking counterparts over the past 30 years, though they have both risen at an astounding pace.

Thirty years ago, in 1983, the average pay of the leaders of the six largest banks was 40 times the average of all U.S. workers, while the multiple for the executives of the nonbank Fortune 50 companies was only slightly less at 38 times. Since then, large-bank CEO compensation has grown geometrically compared with that of the average worker and now stands at 208 times, while the nonbankers average 224 times. Put another way, the average pay for all American workers has grown 2.9 times over the past 30 years, while that of the bank executives grew 15.4 times and the nonbankers by 17.4 times.

When the average worker was earning $23,556, and the top corporate leaders were earning $888,164, compensation packages were not media events like they are today with paychecks of $10 million to $50 million; and it is not unusual or surprising to see such salaries covered widely in the press. Consultants, engaged by boards of directors to advise on pay packages, are only too happy to justify this sky-high compensation through so-called peer group analysis.

Public scrutiny is exacerbated by the earnings of hedge fund managers on Wall Street, where the top 25 averaged $565.6 million and the top 10 averaged over $1 billion each in 2012. While hedge funds are not banks, the man or woman on the street lumps them together, which increases their antipathy toward our industry, particularly when these huge earnings are placed alongside the fines and sanctions the banks have incurred.

In 2012, one American chief executive had a total compensation package of $96 million, while another, whose company had yet to make any money, earned $78 million. A third has a pension package of $159 million, which was recently reported by The Wall Street Journal to be "by far the largest pension on file for a current executive of a public company, and almost certainly the largest ever in corporate America."

These kinds of figures are unfathomable to the average citizen. One can clearly see the American social fabric is at stake; outsize compensation based on limited personal risk but great public exposure invites resentment and, not unjustified, populist anger. The compensation practices of corporate America cry out for correction, preferably through the actions of executives and boards of directors themselves.

We ignore the palpable public outrage over executive compensation and its role in worsening income inequality  the divide between rich and poor  at our peril. Our so-called recovery since the crisis has done little for ordinary citizens. According to the latest U.S. Census Bureau report, there were 46.5 million people living in poverty, representing a poverty rate of 15%  some 2.5 percentage points higher than in 2007. Since the beginning of the crisis, 4.8 million homes have been foreclosed. Even today, 2 million families are seriously delinquent on their mortgages.

When one contrasts these harsh realities with exorbitant executive compensation, which omnipresent media reports regularly highlight as a symbol of excess, it is easy to understand the widespread public resentment. It is this sentiment that provides fuel for our elected officials and policy makers to call for more legislation and regulation, which is placing a costly burden on the engines of growth for our economy.

As we strive to create the framework for the safest, most stable financial system in the world, we must be mindful of the importance of protecting the fiscal interests of taxpayers as well as the dynamism and entrepreneurial spirit of our economy. In this regard, a measure of self-restraint on corporate compensation would seem to be a small price to pay.

Robert Wilmers is the chairman and chief executive of M&T Bank in Buffalo, N.Y. This article was excerpted from his 2014 letter to shareholders.