Twenty-five years ago Makis Voridis was an axe-wielding fascist who patrolled the streets of Greece in hot pursuit of leftist fellow students.

Not much later, after his expulsion from Athens University's law school, he headed the youth wing of Epen, the far right party founded by the imprisoned former dictator Georgios Papadopoulos.

With a seat in the European parliament the group enjoyed close ties with Jean-Marie Le Pen, the former leader of the National Front in France, openly espousing many of his extremist views.

Today Voridis is a senior member of Greece's coalition government, embraced by the business community as the near-bankrupt country navigates its worst crisis in modern times.

"Some bad things were done in the past but we've all put water in our wine," says Alexandros Xenakakis, who has long worked with the politician. "We're not Nazis. We're patriots who care about our nation."

In an interview with the Guardian, Voridis said: "I was a rightwing student activist and, yes, it's true that when I was the head of the Epen youth I had ties with Le Pen who we brought to Greece and, yes, I agreed, not with all but with some of his views.

"But am I a crypto-fascist, with a hidden agenda who wants to abolish democracy and human rights? The answer is no. And do I have a problem with Jews and homosexuals? No, I don't. I'm terribly OK [about that]. Sexuality for sure is a personal choice," insisted the politician whose best man, Carl Lang, was deputy president of the National Front before he parted ways with Le Pen.

The rise of the far right – in government for the first time since the collapse of military rule almost 40 years ago – led to opponents taking to the streets and leftist politicians decrying the development as a "historic mistake". Jewish groups have called the inclusion of the extremist Popular Orthodox Rally (Laos) in the three-party administration "deeply troubling".

Voridis' party boss, Giorgos Karatzaferis, a former bodybuilder who formed Laos in 2000, has been criticised for making a succession of anti-Semitic remarks on his own television channel, Telecity. At the founding congress of his party, he said: "They say that to get ahead you have to be one of three things: a Jew, a homosexual, or a communist. We are none of these ... Vote for a parliament without Masons, without homosexuals, without those dependent on Zionism."

Karatzaferis, a high-school dropout who ran a modelling agency before going into politics, has denied being anti-Semitic and has attributed his inflammatory remarks to getting carried away in his dealings with an insistent media.

"[His] participation besmirches the respectable image of a country known for being the birthplace of democracy," said Serge Cwajgenbaum, general secretary of the European Jewish Congress.

"I am not dogmatic," the 64-year-old said after the new government headed by Lucas Papademos, a technocratic economist, was sworn in last month in an attempt to save Greece from default.

Unlike the maverick Karatzaferis, whose ability to flip-flop on policies has seen him increasingly being dubbed a political opportunist, Voridis is disarmingly open about his past.

Now 47 and greying at the temples, the minister of infrastructure and transport, one of four Laos deputies in the government, does not deny he is a reconstructed fascist.

He describes himself now as a "national liberal", and says the enemy is not so much the liberal left but the hundreds of thousands of Muslim migrants who have either succeeded, or will try, to get into the Mediterranean country, now seen as the easiest backdoor entrance into the EU.

But with Laos (which means the people in Greek) also playing on the fears and insecurity of an austerity-weary nation struggling to make ends meet, there are mounting concerns the ideologically eclectic party, which garnered 5.6% of the vote in the 2009 elections, will add to the turmoil stalking the eurozone's weakest link.

Just as Greece's crisis has enabled Voridis to escape his past – and also resulted in the spectacular rise of the left – there are fears it will allow extremists of every hue to extol their dangerous ideas.

Tellingly, an MRB poll released this week revealed the fascist Chrysi Avgi (golden dawn), a party to the right of Laos, picking up 1% of the national vote for the first time.