Frédéric Mitterrand, France's culture minister, last night denied paying underage boys for sex in Thailand, and sought to calm a storm over his support for the film director Roman Polanski.

Both the extreme right and the Socialist party have called for his resignation over his 2005 autobiography, The Bad Life" in which he described paid encounters with "young boys" in Bangkok. It came back to haunt Mitterrand when Jean-Marie Le Pen's Front National looked into his past following his defence of Polanski, who was arrested in Switzerland on a US warrant over a statutory rape charge concerning a 13-year-old girl and dating back to 1977.

Mitterrand was the first to leap to Polanski's defence, calling his arrest "terrible" – something which he and Nicolas Sarkozy's ruling centre-right party later rowed back from. An emotional Mitterrand appeared on France's evening news to acknowledge paying for sex – but only with people "my age", "consenting" and with "no ambiguity". He said they were not minors.

He added: "I absolutely condemn sex tourism, which is a disgrace, I condemn paedophilia, in which I have never participated in any way. The book is in no way an apology for sex tourism, even if one chapter is a journey through that hell, with all the fascination that hell can inspire."

Mitterrand, a nephew of the late president, is a gay activist, writer and former TV presenter whose appointment was seen as a coup for President Sarkozy.

The autobiography has passages describing the narrator's shame and attraction for gay prostitution bars in Asia, and the pleasure and emptiness of paying for sex. When published it was hailed for its honesty. Asked at the time about his use of the term "young boys", Mitterrand said he called all men "boys" and it did not refer to minors. Sarkozy, when he appointed Mitterrand this summer, described the book as "brave and talented".

Asked if paying for sex in Thailand was a mistake, Mitterrand said: "An error, without a doubt. A crime, no." He said he had Sarkozy's full support and would not resign; politicians attacking him were confusing homosexuality and paedophilia.

His reaction to the arrest of Polanski, who has dual French and Polish nationality, was "too emotive", but he had wanted to show that "an artist of international repute wouldn't be abandoned by his culture minister" and would be treated fairly.