Referring specifically to Bank of America and Citigroup , the managing director at Credit Agricole said the institutions are classic examples of banks where shareholders should be able to step in and remove ineffective executives.

Bad corporate governance — and not fundamental weaknesses of capitalism — is at the heart of why so many people distrust banks, said Mayo, who called for accountability rather than more regulation. Mayo is author of the upcoming Exile on Wall Street: One Analyst's Fight to Save the Big Banks From Themselves.

"Let's go bank by bank, company by company, CEO by CEO, chairman by chairman, let's just go down the list...and start cleaning house," he said.

Looking at specific institutions, Mayo pointed to Citigroup Chairman Dick Parsons and said, "I don't know how he still has a job."

"If you haven't performed for a number of years — like a decade — then get out of the major leagues and go back to the minor leagues, and you should be broken up. But I think the shareholders should lead the way," he said. "If you have a company like Citigroup — maybe one or two others out there — that haven't performed over more than a decade, then I say let's break them into more manageable pieces."