Hundreds of refugees forced to leave a UN-run centre in Libya earlier this year, including survivors of the Tajoura detention centre bombing, are among those worried about being cut off from aid in the coronavirus outbreak.

Last week, the UN Refugee Agency (UNHCR) announced it would suspend some activities in Libya, including work at a Tripoli community day centre and a registration centre where new arrivals can sign up for help. UNHCR will also stop making visits to detention centres until staff are given personal protective equipment, though a spokesperson said the agency will increase phone counselling and outreach to refugee community leaders. Both UNHCR and the International Organization for Migration have halted resettlement flights for refugees and migrants globally.

“We are afraid about this disease, we are confined in our home,” said an Eritrean man in Tripoli. “If the disease [comes to] Libya it will be very dangerous, especially for the refugees, because we live densely in one place.” Between six and eight people sleep in each room, he said, because the price of rent is so high.

“The government prevent all the people from leaving their houses, but we need to work to get money for food,” said another man, from Darfur, Sudan.

Libya’s civil war will be a year old on 4 April. Thousands of the refugees and migrants trapped there previously tried to get to Europe by crossing the Mediterranean on boats, but were intercepted and returned by the EU-backed Libyan coastguard. Many fled wars or dictatorships across Africa and have now spent years in detention centres associated with the Tripoli-based Government of National Accord, in cramped conditions that have repeatedly been condemned by human rights groups.

Earlier this year, hundreds of refugees and migrants were effectively forced to leave the UNHCR-established, donor-funded gathering and departure facility in Tripoli. The facility was heralded as an “alternative to detention” when it was opened in late 2018 – intended to be used as a safe place where vulnerable people could be brought ahead of evacuation. Instead, it quickly became mired in problems.

In interviews with the Guardian in Rwanda, refugees since evacuated described how the facility was used as a military base, with weapons stored around the perimeter. They say militia leaders also sheltered there, hoping it would stop them being hit by an air strike – effectively using the refugees inside as human shields.

Some refugees described themselves as “slaves”. They said they were forced to unload and carry heavy weapons and ammunition, and even to build a shelter for horses belonging to Mohammed al-Khoja, a militia leader and deputy head of Libya’s department for combating illegal migration. The department did not respond to a request for comment.

“If you don’t want to [work] they force you to go,” one Eritrean said. “It was a really bad time.”

UNHCR said it is aware of allegations that refugees were used as forced labour in the gathering and departure facility, but staff could not verify this information themselves. A spokesperson said they did not know about weapons being kept there. “The gathering and departure facility remains under the jurisdiction of the [Libyan] ministry of interior, who are responsible for security,” said UNHCR’s Charlie Yaxley.

In January, UNHCR announced it would be suspending work at the centre. “It’s going to become a military target,” said UNHCR special envoy for the central Mediterranean Vincent Cochetel, speaking in Berlin in January. “I prefer that the people go to Tripoli, get assistance. Will some get killed? Yes, that’s Libya, we can’t protect people.”

Conditions for refugees in Libya have been repeatedly criticised by human rights groups. Photograph: Giles Clarke/UNOCHA

Those who left the gathering and departure facility were offered an “urban assistance” programme, which was supposed to include emergency cash. However, some of the payments have not come through, with a UNHCR spokesperson saying “bank liquidity issues in Libya [are] an ongoing problem” but that they made distributions last week and “hope cash distributions can resume in a few days”. Refugees complain many were also never given promised appointments with UN staff. “They said they would contact us in weeks, but one month passed,” one Darfuri said.

Though there are still no cases in Libya, aid workers also worry there could be a backlash against refugees and migrants if Covid-19 spreads.

“There is a real risk of coronavirus being conflated with tuberculosis – which we all know is an issue already – and of migrants being unfairly blamed and stigmatised for it,” said Liam Kelly, the country director for the Danish Refugee Council.

Another aid worker, who asked not to be named, said the outcome for migrants and refugees would be “disastrous” if there is an outbreak. “The restrictions on activities of [the] UN and international organisations may deteriorate the situation for refugees and migrants even further,” he said.

This weekend, roughly 200 refugees and migrants were released from Tripoli’s Sabaa detention centre – many of whom have health problems. In January, the Guardian revealed that a 16-year-old boy incarcerated in Sabaa had died from an unknown illness.

Though the risk of initial transmission may be lower than in urban areas, Covid–19 is a grave threat for the thousands of refugees and migrants still locked up in detention centres. In Khoms’ Souq al-Khamis detention centre, 75 miles (120km) east of Tripoli, an Eritrean detainee said many people used to go out and work during the day in exchange for food. That has now stopped because of the coronavirus. “No one gives us food, we are suffering,” he said. “We don’t know what to do or where to go, I think our story will end here.”

In Zintan detention centre, 113 miles south-west of Tripoli, dozens of detainees have already died from tuberculosis and other medical problems, as well as in a recent fire. Now one detainee said he is more concerned than ever about the lack of space, “because all the people use common toilets and [are] crowded [together].”

Some refugees desperately continue to try and cross the Mediterranean Sea to reach Italy, but humanitarian search-and-rescue missions have been halted since 27 February.

“The rescue ships are not yet back,” messaged a refugee in Tripoli who was hoping to attempt the journey. “The problem is coronavirus.”