The French minister in charge of clamping down on tax evasion resigned on Tuesday after allegations that he evaded taxes with a secret Swiss bank account.

Jérôme Cahuzac, the budget minister, who has vehemently and repeatedly denied the accusations, stood down hours after it was announced a legal investigation had been opened into the accusations.

The launching of legal proceedings for "fiscal fraud" did not name Cahuzac but centres on claims that he hid money from the French tax authorities, first in Switzerland and then in Singapore.

It followed a preliminary inquiry opened in January to determine whether a voice on a recording of an alleged conversation between Cahuzac and his wealth manager in 2000 was indeed that of the minister.

On TuesdayCahuzac continued to insist he was innocent, and lambasted what he said were defamatory accusations. He said he had resigned for the "good of the government".

A spokesperson for François Hollande said the president had accepted Cahuzac's resignation and removed him from office. In a statement, he thanked Cahuzac for his "talent and competence", and said he applauded his decision to resign to "better defend his honour".

French prime minister Jean-Marc Ayrault added that Cahuzac had resigned in order to concentrate on his legal defence and "to not embarrass the government".

"The prime minister regrets but understands his [Cahuzac's] decision and thanks him for his work in government," said a statement from the PM's office.

It is the first serious scandal to hit Hollande since his election last May and a blow to his government, which he promised would be "beyond reproach".

The embarrassment is compounded by the fact that Cahuzac, 60, a former MP and cardiologist who became a hair transplant surgeon, has headed Hollande's efforts to cut its public deficit and stamp out tax evasion.

He had appeared to be in the clear last month after the Swiss authorities said they could find no trace of an account. However, the French investigative website Mediapart, which broke the original story, insisted it had evidence.

Mediapart claimed in January to have evidence that Cahuzac had an undeclared account at the Swiss bank UBS for 20 years until 2010. The website alleged he travelled to Switzerland to close the Geneva-based account and transfer the money in it to Singapore shortly before he was made head of the parliamentary finance commission in February 2010.

An alleged recording of a conversation between Cahuzac and his wealth manager in 2000 about his embarrassment over the Swiss account was published by Mediapart.

"What bothers me is that I still have an account open with UBS … UBS is not necessarily the most hidden of banks," he is alleged to have said.

Cahuzac insisted the voice in the recording was not his, and announced he was suing Mediapart for defamation. "I do not have, I have never had a foreign [bank] account. Not now. Not before," he told MPs in December.

During his successful presidential campaign, François Hollande insisted his government would be "beyond reproach".

The editor of Mediapart, Edwy Plenel, has publicly insisted he stood by the story. "There is one question we need to ask our government, prime minister and president. Can the budget minister, the head of the tax administration, carry out his work properly and legitimately when there is enough evidence to open a preliminary investigation?" he said in January.

On Tuesday Mediapart said it had been vindicated: "For more than three months now, Mediapart has put forward information that has today been confirmed, validated and verified by the prosecutor's office in Paris and investigators," Plenel said.

"The real question is why Monsieur Cahuzac lied with such aplomb? Did he lie for himself? Did he lie for others?"

One or several independent investigating judges will be appointed in the next few days to delve into the Mediapart recording. Europe1 radio said three witnesses who had heard the recording had recognised Cahuzac's voice.

The prosecutor's office said in a statement: "The investigations carried out as part of the preliminary inquiry must continue from now in a more appropriate form", saying the opening of an official inquiry would enable detectives to officially request information from Switzerland and Singapore.

Thierry Mandon, spokesman for the Socialist group in the Assemblée Nationale, said Cahuzac's resignation was "unavoidable".

"It was unavoidable in the sense that its a good point of law that those who have to go and defend themselves before the court can concentrate on their defence," Mandon said.

"We have to move fast but at the same time we have to respect the person who must now defend himself in the court."

Cahuzac was replaced as budget minister by the European affairs minister, Bernard Cazeneuve.