Bank of England governor Sir Mervyn King says the UK banking industry requires a change in culture Press Association

Sir Mervyn King, governor of the Bank of England, has piled the pressure on the scandal-ridden City when he said something had gone "very wrong" with Britain's banks that needed to be put right.

As Barclays and other high street banks became embroiled in a new misselling scandal, King launched his most scathing attack yet on the culture of banking in the five-year long financial crisis.

King refused to say Bob Diamond was a "fit and proper" person to run Barclays as the reputational damage from an interest rate fixing fine led to another fall in the bank's shares. More than £4bn has been wiped off the value of the bank since the rate-fixing scandal emerged.

"It is time to do something about the banking system", the governor said.

As he warned the outlook for financial stability had deteriorated as a result of the deepening crisis in the eurozone, King dismissed mounting calls for a Leveson-style investigation into banks saying that enough was already known to implement root and branch reform of the City.

But Ed Balls, the shadow Chancellor, said such an inquiry was needed. "The governor is right to say that there is a real cultural problem in our banks which these latest scandals expose and I don't think it is right for government ministers or for the governor, actually, to say we have had enough inquiries," said Balls.

King said: "Many people in the banking industry are hard-working and feel badly let down by some of their colleagues and leaders.

"It goes to the culture and the structure of banks – the excessive compensation, the shoddy treatment of customers, the deceitful manipulation of a key interest rate, and today news of yet another mis-selling scandal."

Barclays, HSBC, Lloyds Banking Group and Royal Bank of Scotland were yesterday ordered by the Financial Services Authority to pay redress to small business customers after finding "serious failings" in the way they were sold complex financial products intended to protect them from interest rate rises. The revelations caused outrage among business groups with the Institute of Directors, the British Chambers of Commerce and the CBI all condemning the fleecing of their business members. John Longworth, director general at the BCC, said the latest scandal would "damage business' perception of banks further".

After 72 hours of sustained attack on Diamond, the Barclays boss made clear he would not voluntarily step down. The bank's chairman, Marcus Agius, is also in the firing line and may be forced out in an attempt to show that a boardroom executive is paying the price for the record-breaking £290m fine and damage to its reputation. Barclays shareholders are thought to be considering appointing a figurehead to conduct a review of the culture inside Barclays.

The political pressure continued to mount on Diamond. Asked whether Diamond was the right man to lead Barclays, the prime minister said: "I can't say that. He has questions to answer."

Cameron's remarks mirrored those of King and Lord Turner, chairman of the FSA, who both declined the opportunity twice to say that Diamond was "fit and proper" to run Barclays.

Barclays was on Wednesday fined a record £290m for attempting to manipulate crucial interest rates known as the London interbank offered rate (Libor) and the Euro interbank offered rate (Euribor) between 2005 and 2009 both of which are crucial in setting the price at which customers borrow for the mortgages.

The fine was published alongside a series emails showing how traders helped each other to try to manipulate interest rates with promises of Bollinger champagne and quips like "this is for you, big boy."

Sir Roger Carr, president of employers' body CBI, said: "The manipulation of the Libor arrangements is deplorable and undermines international trust in the integrity of the City. The weakness must be addressed and the culprits punished".

King added that the question of who ran the UK banks was a question for another day, insisting that the immediate priority was for the government to implement in full the recommendations of the Independent Commission on Banking headed by Sir John Vickers, which called for firewalls to be set up between the investment and retail arms of banks. The governor said the cultures of investment and retail banking were completely different and needed to be separated.

Turner said that there was a "culture of cynicism and greed that is quite shocking." Andrew Bailey, the top banking regulator at the FSA, said the onus was on bank boards to take action. "If, as we now see, there is a fundamental breakdown in trust, the bank boards have to recognise that trust has to be got back and they have to think very hard about how they do that."

The prime minister agreed with King's calls for a cultural change in banking. "We know what's gone wrong and largely we know what needs to be done to put it right. What you are going to see from the government is an incredibly methodical series of actions to deal with all of these problems," he said as he rejected the call by Balls for a Leveson-style inquiry. "People are shocked by the swaggering arrogance of what we have discovered in the last 24 hours. We really need to open this wide open," said Balls.

Knowing he was opening himself up to political backlash as he was City minister during part of the Labour government, Balls admitted mistakes had been made.

Lord Oakeshott, the Liberal Democrat peer, said Balls and the former prime minister Gordon Brown would need to be the first two witnesses called to give evidence.