(CNN) -- It's a community of crude green metal sheds, baking hot during the day and lacking clean running water. Residents say thieves can slice through the thin walls with their knives.

A man packs up his belongings after being evicted from Borei Keila to Tuol Sambo.

Locals call it "the AIDS colony."

The Cambodian government set up the community on a site known as Tuol Sambo, on the outskirts of the capital, Phnom Penh. It began relocating families affected by HIV and AIDS from the neighborhood of Borei Keila last month, and last week removed 20 more families.

The move sparked outrage among human rights groups, which said the conditions in Tuol Sambo do not meet standards for even emergency housing, and that the isolation would further stigmatize those affected by HIV.

"By bundling people living with HIV together into second-rate housing, far from medical facilities, support services and jobs, the government has created a de facto AIDS colony," said Shiba Phurailatpam of the Asia-Pacific Network of People Living With HIV.

Human Rights Watch released a letter Tuesday that was signed by more than 100 global HIV/AIDS and social justice organizations and sent the day before to Cambodian Prime Minister Samdech Hun Sen and Health Minister Mam Bunheng.

"We are deeply disturbed by the Cambodian authorities' creation of a de facto AIDS colony at Tuol Sambo," the letter said. "Tuol Sambo is far away from the jobs, medical facilities and support services that had been available to Borei Keila residents in the city."

The letter said HIV-affected families are being moved to unsafe housing while, just a short distance from "the AIDS colony," nonprofit agencies are helping build brick houses for other homeless families -- ones without HIV.

The letter acknowledged the international recognition the Cambodian government has received for treating and supporting people living with HIV. But it called the Tuol Sambo relocations "discriminatory" and "life-threatening" to people whose immune systems are compromised.

The evictions took place despite repeated appeals to the government, including from the United Nations, the human rights activists said.

"We have reason to fear that relocations of HIV-affected families are continuing even as we sign this letter," they said.

Reports compiled by the United Nations show that HIV is declining in Cambodia. In 2008, an estimated 67,200 adults and 3,800 children were living with HIV. A resurgence of an epidemic exists, however, according to UNAIDS.

The agency reported a risk of second-wave HIV infections among high-risk populations, such as female sex workers, their clients and their partners.

The HIV/AIDS families relocated by the Cambodian government worked mostly as day laborers, motorcycle taxi drivers, seamstresses and cleaners, the human rights groups said. They said most of those relocated have little prospect of finding work in Tuol Sambo.

Government officials told The Phnom Penh Post that they knew about the problems at Tuol Sambo and that steps were being taken to improve conditions.

"We are trying to find clean water for them," Phnom Penh's Deputy Governor Mann Chhoeun told the English-language newspaper. "They will also no longer face problems getting medicine, because we have given one house to [Center of] Hope, which has improved the health of the village."

A mission of an area hospital, The Center of Hope provides free medical care for the poor. But Gerlinda Lucas, the deputy director of grant management, told the Post that the center had no plans to open a permanent clinic at Tuol Sambo, though it will continue weekly mobile clinic service.

Meanwhile, the relocated families expressed relief that they were leaving a neighborhood where landlords refused to rent to them. But they worried about their safety after thieves cut through the metal walls of one resident's home.

"I do not feel safe here because our houses are not strong," Chheang Toma told the Post.

And then, there was the added fear of starting anew in an unfamiliar community.

All About Cambodia • Phnom Penh • HIV and AIDS