Story highlights Five countries have all requested bailouts -- Greece, Ireland, Portugal, Spain and Cyprus

Katainen said the crisis is serving as a "very hard lesson" to many eurozone countries

A banking union regulated by the European Central Bank that will become active in 2014

Eurozone nations must follow the rules if they want to get out of the financial crisis plaguing the region, says Finnish Prime Minister Jyrki Katainen.

In an interview with CNN's Richard Quest, Katainen said the crisis is serving as a "very hard lesson" to many countries inside the 17-nation single currency union.

"One of the reasons why we are in crisis at the moment is that everybody did not follow strictly the rule base and now we pay the price," Katainen said.

Since the eurozone debt crisis began in 2010, five countries -- Greece, Ireland, Portugal, Spain and Cyprus -- have requested bailouts from Europe's rescue funds.

But Katainen said a raft of new regulations within the next few months will make Europe "much better, safer and sustainable."

JUST WATCHED Finland delivers warning to Cyrpus Replay More Videos ... MUST WATCH Finland delivers warning to Cyrpus 01:43

This includes a banking union regulated by the European Central Bank that will become active in 2014 and the euro-wide rescue fund known as the European Stability Mechanism.

Katainen highlighted falling interest rates of government debt for countries such as Ireland and Portugal as a sign that European confidence is returning. He added: "I am a former finance minister and I have learned not to say that we have seen the worst."

The Finnish Prime Minister, who in the past has been an outspoken supporter of austerity for countries that have spent recklessly, said European nations must restore competitiveness to their economies.

"In Spain the competitiveness has been improving a lot they have done labor market reforms, they have done similar things in Ireland but it always takes some time to function."