Migrant workers building a multibillion-pound cultural hub in the United Arab Emirates, which includes new Guggenheim and Louvre museums, are subject to destitution, summary arrest and deportation if they complain about their squalid and unsafe conditions, an investigation by Human Rights Watch (HRW) has found.

Its new report on Saadiyat Island in Abu Dhabi found that some workers were subjected to conditions amounting to forced labour, having had their passports confiscated and been so poorly paid they struggled to pay off recruitment fees which were supposed to have been banned. Attempts to raise concerns about the workers’ mistreatment led to their wages being withheld, arbitrary police intimidation and their forcible removal from the country.

As Abu Dhabi gets a multibillion-pound cultural facelift, many of the migrant workers from Bangladesh and Pakistan complain of appalling conditions Guardian

Although it acknowledged some progress had been made, the report, published on Tuesday, disputes claims by the UAE authorities and western organisations involved in the cultural hub – which includes New York University (NYU) and the British Museum – that effective measures have been put in place to safeguard workers’ rights. Researchers found that legal reforms to allow workers to change employers without their consent and to revoke the licences of agents who charge workers recruitment fees were not enforced, and that workers were unable to file grievances or were threatened into staying silent.

Interviews with more than 100 workers in 2013 and 2014 revealed endemic abuse of workers’ rights consistent with HRW reports in 2012 and 2009, and investigations by the Observer and campaign group Gulf Labour. Contractors failed to pay wages for months at a time, did not renew work permits and residence visas, and refused to pay the end-of-service benefits to which workers were entitled, leaving them destitute. All of those interviewed said employers had withheld their passports and had failed to reimburse the recruitment fees they were forced to pay to secure their jobs.

The report comes as the UN investigates the abuse of migrant workers in the UAE. The investigation by the International Labour Organisation (ILO) comes after a complaint brought by the International Trade Union Confederation, which highlighted poor conditions on Saadiyat.

HRW suggests that the UAE is complicit with the exploitation of workers on Saadiyat. Fifteen former workers on the NYU and Louvre construction sites, who were deported in 2013 and 2014 following strikes over low pay, said the contractors who employed them acted “in concert with the authorities, who arbitrarily detained and deported scores of workers”. Two former workers on the NYU site said police officers had slapped their faces during interrogations in a bid to force them to name strike leaders. A worker at the Saadiyat accommodation village, where all the migrant workers building the flagship museums are supposed to live, said about 500 men living there had been deported or had their work visas cancelled after one of the strikes.

One worker on the Louvre site said he was owed more than £1,250 ($1,900) in unpaid wages and end-of-service benefits, dating back to 2005. Neither he nor a colleague in a similar situation had filed a complaint after being threatened by their employer. A group of 12 Louvre site workers had filed a complaint for unpaid wages against the same contractor in a Dubai court in February, but the case had been adjourned twice and remained unresolved. Meanwhile, three Bangladeshi workers on the NYU site said they were on a basic salary of £125 per month – half the amount they had been promised – but they could not leave the country as they were still paying off the debts they had accrued to gain their employment, including £1,690 in recruitment fees.

Many workers were found to be living in slum-like accommodation. In Abu Dhabi city centre, a researcher found 27 men crammed into two small rooms, including 11 who worked as painters at the NYU campus. Fifteen men lived in one room and 12 in the other. Video footage shows insects crawling around the kitchen, exposed electrical wires wrapped around a showerhead, and a hole punched in the fire escape door, which was locked.

The report calls on the UAE to pass legislation that expressly criminalises passport confiscation and stops striking workers being deported. It also urges the western institutions involved in the project to ensure that safeguards to protect workers are enforced, as well as to compensate mistreated workers.

Nicholas McGeehan, HRW Gulf researcher, said hundreds, if not thousands, of workers had been scammed into accruing debts they cannot repay and those who protested were “deported like cattle back home, some of them without their shoes”. He added that those who remain in the UAE were now “working under the menace of penalty – in the knowledge that if they withdraw their labour, they could be beaten up by special police in balaclavas. Why haven’t the French government or New York University condemned that?”

The UAE government has not responded to the report. Two HRW researchers were blacklisted last year and told they could never return to the country while another was barred from re-entering. Abu Dhabi’s Tourism Development and Investment Company (TDIC), which oversees the construction of Guggenheim and Louvre museums, and the Zayed National Museum, to which the British Museum is a cultural adviser, said the findings did not “truly reflect the reality on the ground”.

The Guggenheim said in a statement that it was encouraged by TDIC’s “continued commitment to advancing enforceable, measurable protections for workers’ rights” on Saadiyat.

It acknowledged that there remained room for improvement, but added that representatives from Guggenheim who had visited Saadiyat had “observed firsthand its elevated standards and TDIC’s attention to continued improvement”.

The most recent audit of Saadiyat Island by PricewaterhouseCoopers, which was appointed by TDIC to monitor workers’ welfare, found that the company had not consistently enforced its labour policies. According to the audit, TDIC imposed financial penalties against only three of the six contractors who were found to be in breach of its employment policies in 2014. However, the audit noted that TDIC has also pledged to introduce a fairer employment code this year and has already made some improvements to workers’ conditions, including door-to-door laundry services and new kitchen facilities at the Saadiyat accommodation village.