Tokyo(CNN) Japan has upheld a law effectively requiring trans people seeking to legally change their gender to be sterilized.

Takakito Usui, a transgender man who wants to change the gender listed on his official documents, had appealed to the court seeking to overturn Law 111, which requires applicants to "permanently lack functioning" reproductive parts to qualify for gender affirmation.

The Supreme Court unanimously rejected Usui's case Thursday, ruling the 2003 law constitutional -- though judges added it was invasive and encouraged the legislature to review it.

"It is unthinkable in this day and time that the law requires a sex-change operation to change gender," Usui's lawyer Tomoyasu Oyama told CNN.

"When the law was established 15 years ago, LGBT people had to make a bitter decision and swallow the conditions to pave a narrow way for official change of gender. With this decision, I hope lawmakers will change the law to support the wishes of the LGBT community."

The court initially said the law was intended to prevent "problems" in parent-child relations which could lead to societal "confusion," and avoid "abrupt changes" to society. About 7,000 people have changed their gender registration under the law since it was first passed.

While they ruled against Usui, two judges issued an additional opinion calling on society to "embrace the diversity of sexual identity."

Suki Chung, Asia Pacific campaign manager at Amnesty International, said the ruling was "a blow for the recognition of transgender people in Japan. It is a missed opportunity to address the discrimination transgender people face."

"Forcing people to undertake medical treatment in order to obtain legal gender recognition violates their right to the highest attainable standard of health. We urge the Japanese government to end this discriminatory and highly intrusive policy," she told CNN.

People attend the annual Tokyo Rainbow Parade in May 2018. Japan has made limited progress on LGBT rights, but activists warn significant hurdles remain.

'Stain on Japan's record'

Requiring sterilization has been widely denounced by LGBT groups in Japan and around the world, and in 2013 the UN's special rapporteur on human rights called on all states "to outlaw forced or coerced sterilization in all circumstances and provide special protection to individuals belonging to marginalized groups."

In a 2017 report noting some advances on transgender rights, Human Rights Watch said the 2003 law "remains a stain on Japan's record."

"The procedure is discriminatory, requiring applicants to be single and without children under 20, undergo a psychiatric evaluation to receive a diagnosis of 'Gender Identity Disorder', and be sterilized," the report said.

Nine countries across Asia provide no right for people to change their legal gender, according to NQAPIA, a federation of regional LGBT groups, while a number of others have significant hurdles to doing so.

Conservative Japanese lawmaker Mio Sugita, who belongs to the ruling Liberal Democratic Party, attracted widespread criticism last year when she published an article saying "Support for LGBTs has gone too far."

"Will people agree to have their taxes used on LGBT couples? They cannot have children, so they are unproductive," said Sugita, according to the Japan Times.

However polls suggest Japan is becoming less conservative on LGBT issues. A poll this month by advertising firm Dentsu found more respondents than ever openly identified as lesbian, gay, bisexual or transgender.

Over 70% of respondents said they supported stronger legal protections for LGBT people.