It's the topic of the moment and everyone seems to agree — Europe needs fewer banks. Profitability in the European banking sector has never fully recovered since the financial crisis 10 years ago, and several countries have been slow to impose restructuring or tackle non-performing loans. The overall number of banks and credit institutions in the European Union decreased to 6,648 in 2016, from 8,570 in 2008, according to Statista. However, industry observers say the pace of reduction and amalgamation needs to increase further.

The European Central Bank (ECB) is now spearheading a banking union project to make Europe's bloated financial sector more resistant to shocks. Speaking to CNBC's "Squawk Box Europe" on Thursday, Jose Vinals, group chairman of Standard Chartered, said the ECB's mandate as a financial supervisor should speed consolidation. "I think a lot is changing with the creation of the banking union and with the ECB becoming the main authority in charge of supervising banks in Europe. They are taking a much stronger approach," he said. Vinals added that to make such a banking union work properly, it would be very important to have cross-border banking mergers and acquisitions (M&A), saying, "It hasn't happened yet, to an extent that it is healthy." Speaking at the Handelsblatt banking conference in Frankfurt, Germany, on Wednesday, ING Chief Executive Ralph Hamers told CNBC's Annette Weisbach that while consolidation in Europe had its merits, regulators would need to further loosen cross-border rules. "The real benefit is only on the cost side. Capital is not freely transferable, liquidity is not freely transferable. So, for things to happen cross-border, probably you need more to make it attractive," he said.

Spain can lay claim to being further along the road of consolidation than most European countries. Jose Maria Roldan, chairman and CEO of the Spanish Banking Association, said Thursday that the country has seen its number of savings banks dwindle from 45 in 2008 to 18. Roldan told "Squawk Box Europe" that the ECB and banking supervisors were spelling-out lean times for participants in the sector. "The message that we are getting across from the ECB and the single supervisory mechanism is that going forward, profitability pressures will continue," he said. "This is because of digital transformation, fintech competition, and because the normalization of interest rates is going to be slower than in the past." Roldan said the ECB's underlying point to banks in Europe was if you can't return to profit then "For Sale" signs should be put up. "I think the message that is being sent across is that if you cannot guarantee profitability with your business model then you have to think about mergers as a possibility for you," he said.