"As a consequence, the financial industry has largely lost the public trust," Dudley said, according to remarks released by the New York Fed.

At an otherwise docile gathering Monday, New York Fed President William Dudley and Fed Gov. Daniel Tarullo delivered a message to high-ranking Wall Street executives that bad behavior won't be tolerated. Dudley in particular said big financial institutions operate under a cloud of suspicion and have continued bad behavior even after suffering more than $100 billion in fines and triggering a multitrillion-dollar bailout after the financial crisis.

The mood may have been polite and diplomatic, but the message was not.

The luncheon featured dining on chicken and salmon and had an atmosphere that was "collegial, probably a little too collegial," according to someone quoted by the Journal.



Read More



Ethical transgressions are certainly nothing new to Wall Street, but the crisis and its tales of bankers' taking advantage of consumers and investors—"muppets," as one Goldman trader infamously referred to them—have brought greater urgency to the issue.

Bankers reportedly have talked about setting up an ethical standards board along the lines of what exists in the U.K., but Wall Street has balked.

Things need to change soon, Dudley warned.

Read MoreNow Steve Cohen is paying for being extra ethical

"A number of factors have contributed to the cultural failures that we have seen. An important question is whether the sheer size, complexity and global scope of large financial firms today have left them 'too big to manage,' " he said.



"Financial firms exist, in part, to benefit the public, not simply their shareholders, employees and corporate clients. Unless the financial industry can rebuild the public trust, it cannot effectively perform its essential functions," he added. "For this reason alone, the industry must do much better."



Dudley suggested compensation "can play a useful role in reducing excessive risk taking," an idea floated recently by the International Monetary Fund, which said pay should be tied to responsible risk management.

Read MoreIMF: Time has come for big changes in banker pay

Failure to implement reforms would have dire consequences, he added.

"In conclusion, if those of you here today as stewards of these large financial institutions do not do your part in pushing forcefully for change across the industry, then bad behavior will undoubtedly persist," Dudley said. "If that were to occur, the inevitable conclusion will be reached that your firms are too big and complex to manage effectively. In that case, financial stability concerns would dictate that your firms need to be dramatically downsized and simplified so they can be managed effectively."