The Japanese government must halt the return of women and children displaced by the March 2011 nuclear disaster back to areas of Fukushima where radiation levels remain high, a U.N. human rights expert said Thursday.

The special rapporteur on hazardous substances, Baskut Tuncak, also criticized in his statement the government’s gradual removal of evacuation orders for most of the radioactive areas as well as its plan to lift all orders within the next five years, even for the most contaminated areas.

“The gradual lifting of evacuation orders has created enormous strains on people whose lives have already been affected by the worst nuclear disaster of this century. Many feel they are being forced to return to areas that are unsafe,” he said.

An official of Japan’s permanent mission to the international organizations in Geneva rebuffed the statement, saying it is based on extremely one-sided information and could fan unnecessary fears about Fukushima.

Tuncak expressed concerns about people returning to areas with radiation above 1 millisievert per year, a level previously observed by Japan as an annual limit so as to prevent risks to the health of vulnerable people, especially children and women of reproductive age.

“It is disappointing to see Japan appear to all but ignore the 2017 recommendation of the U.N. human rights monitoring mechanism to return back to what it considered an acceptable dose of radiation before the nuclear disaster,” he said.

In the wake of the Fukushima reactor meltdowns, the Japanese government heightened the annually acceptable level of radiation to 20 millisieverts, raising concerns for the health of residents.

In August, Tuncak and two other U.N. human rights experts jointly criticized the Japanese government for allegedly exploiting and putting at risk the lives of “tens of thousands” of people engaged in cleanup operations at and around the crippled Fukushima No. 1 nuclear plant, a claim Tokyo dismissed.