Dozens of people who were put under pressure to leave a failing EU-funded UN refugee agency centre in Tripoli have used smugglers to cross the sea to Italy in the last month, according to refugees and aid workers.

One Somali man who spoke to the Guardian said he was among a group of more than 50 who left Libya in November and December and were rescued by ships including the Ocean Viking, the Médecins Sans Frontières and SOS Méditerranée ship, in the Mediterranean. He is now in Italy.

Those rescued with him included pregnant women and children. The group were moved to the Tripoli UNHCR centre following the Tajoura detention centre bombing six months ago, he said. Within months they allegedly came under pressure to leave again, as the centre became overcrowded and the UN rejected them for evacuation.

“Now I cannot hear the name UNHCR because they do not help people,” said the Somali man in Italy, who – like other refugees interviewed – asked not to be named due to security fears. He said the people remaining in the UN centre are the most vulnerable – sick people who can’t take to sea or people without money. “Those who cannot run and walk, the children, those with tuberculosis,” he added.

The group are among a number of people that the UNHCR says have agreed to leave the centre while accepting an “urban assistance package” intended to support refugees and migrants living in urban areas of Libya. The Guardian has interviewed more than a dozen refugees who have taken this package. All said it was inadequate and that they remain desperate for safety.

The UN in Libya has been heavily criticised for complying with EU migration policy, which involves funding the Libyan coastguard to intercept boats containing refugees and migrants trying to reach Europe.

Upon return to Libya, many are locked up indefinitely in a network of centres associated with the UN-backed Government of National Accord’s Department for Combating Illegal Migration (DCIM); in reality, most are run by militias.

Detainees are subject to a wide range of human rights abuses, including sexual violence, physical abuse, deprivation of medical care, food and water, and – since war started in Tripoli last April – forced recruitment into militias and the use of refugees as “human shields”.

When questioned on this policy, spokespeople said the EU is working to improve conditions for returnees by funding the UN.

Detainees in detention centres in Libya have reportedly been subject to human rights abuses. Photograph: Taha Jawashi/AFP

The UN gathering and departure facility (GDF) opened in December 2018 as an “alternative to detention”. Initially called “the hotel” by refugees who praised it as the best place they had stayed in years, it has been mired with problems, including growing control by militia leaders, who are reportedly profiting from it.

In December, the Associated Press reported that millions of euros in contracts for food and aid for the centre went to a company linked to militia leader Mohammed al-Khoja, who is also the deputy head of DCIM. The UNHCR says this year it will contract directly for services, rather than through the Libyan government agency LibAid.

In November, the Guardian reported that the UN was planning to withdraw food for everyone left inside the GDF – including women, children, and survivors of the Tajoura bombing. Another group of roughly 400 people formerly held in Abu Salim detention centre were already “starving” in the GDF, according to an internal UN document, and were surviving by eating food shared by other refugees. At the end of December, a wall was reportedly built to block them off from everyone else. Some who tried to leave the centre to buy food in the following days were allegedly arrested.

“The situation is seriously difficult,” an Eritrean there said via text message. “Everyone is suffering and stressed.”

The UNHCR said it could not confirm the arrests. In an email, spokesperson Cecile Pouilly said the GDF falls under the jurisdiction of the Libyan Ministry of Interior and said she believes the wall was built because a training centre is being set up on the centre’s perimeter.

“UNHCR has continuously stressed the need to preserve the civilian character of facilities and infrastructure sheltering civilians, and raised its concerns with the authorities, since such a training centre for DCIM personnel could become a military target,” she said.

At least 30 people were killed and 33 wounded in an attack on a military training centre in Tripoli on Saturday, highlighting the danger.

Last Thursday, three mortars hit very close to the GDF. “It’s too hard, there’s bombing [with] mortars and rockets, they’re shooting around us in the gate of the GDF and almost 200 metres from the GDF,” said an Eritrean man there.

“How in the world, how can this happen?” he asked. “GDF is the centre of civilians so most people here are civilians. That’s why we need this place [to be] safe from others.”

“The people are afraid to sleep tonight inside the buildings,” said a Sudanese man. “But the weather is very [cold] and rain is falling.”

The Tajoura detention center, which held at least 600 people, was bombed in July 2019. Photograph: STR/EPA

That same afternoon, survivors of the Tajoura bombing held a memorial in the GDF to mark the six-month anniversary of the airstrike that killed at least 53 detained migrants and refugees. They sang songs, made speeches, and held posters. One read: “We never demanded [a] special country, but anywhere we will be safe.”

Some survivors wore T-shirts painted with pictures of bloody limbs – a memory of what they saw the night of the bombing.

“Still we are vulnerable,” said an Eritrean survivor who took part in the memorial. “Our brothers died, we lost them. Their destination was to live a safe life … But they were lost with the airstrike.”