European banks that are focused on one specific market or have limited cross-border activities will likely fare better than those that don't, the CEO of Austrian banking group Erste told CNBC Monday.

Since the global financial crisis of 2008 and the euro zone sovereign debt crisis of 2011, European banks have struggled to improve their balance sheets. According to Andreas Treichl, the CEO of Erste, they still have a long way to go before being in a "good shape."

"The issue is that the European financial system is just not in a really good shape yet," Treichl told CNBC's "Street Signs."

He noted that Scandinavian banks and others that have limited activity across Europe are in a stronger position. "If you look at certain banks, in certain countries, they are doing extremely well," he said.

"Most of them are actually non-cross-border banks. You look at Scandinavian banks or banks that are located in Spain but only have foreign business outside of Europe. They do a lot better than those banks that play a game on European convergence and that should give a really good story to European politicians," Treichl said.

European lawmakers have tried since the global financial crisis to reinforce rules and to integrate European banks further. Though the so-called Banking Union has evolved over the last couple of years, it is not yet complete. Some member states are cautious about integrating their banking systems with others in Europe given the discrepancies across the euro zone.