Philippine President Rodrigo Duterte has voiced his support for same-sex unions, telling an LGBT gathering that he even toyed with the idea of becoming bisexual so he could “have fun both ways.”

Duterte made the remarks at an LGBT gathering at Azuela Cove in Davao City on Sunday.

“I said I am for (same) sex marriage if that is the trend of the modern times,” he said. “If that will add to your happiness, I am for it.”

Duterte said he was against gender discrimination, and that he had two gay brothers-in-law. The president also voiced his support for having gay members in his cabinet, adding that he was open to allowing same-sex marriage, the Philippine Daily Inquirer reports.

“Why impose a morality that is no longer working and almost passé. It’s leftover rice,” he said, in a likely reference to the Catholic Church’s stance on the issue. “So I am with you.”

The controversial leader’s statements may cause uproar in the largely Catholic country, however. Duterte is at odds with the Catholic Church on a number of issues, including his support for artificial birth control and the reinstatement of death penalty.

“I want gay marriage. The problem is we have to change the law, but we can change the law,” he said, referring to the country’s Family Code, which says marriage is between a man and a woman. Duterte even said he once considered being bisexual, so he could “have fun both ways, but that did not happen.”

He also told the crowd, “When I was in high school, I did not know if I wanted to be a girl or a boy.”

Before his election as president, Duterte expressed his support for LGBT rights, telling a talk show in 2015 he would not mind if his son came out as gay, as “everyone deserves to be happy.” He also said he would consider legalizing gay marriage and let gay people serve in the military.

However, in March this year Duterte said the Philippines should not have to adopt the West’s mindset regarding gender and gay marriage. Referencing a Time magazine article on the blurring of gender in the US, Duterte told a gathering of Filipinos in Myanmar, “That’s their culture. It does not apply to us. We are Catholics and there is the Civil Code, which says that you can only marry a woman for me… for a woman to marry a man,” The Phil Star reports. “I have nothing against gays.”

Human Rights Watch issued a statement in response, saying the Philippines should adopt gay marriage.