Two-thirds of the population of the Manus Island detention centre are participating in a hunger strike as detainees grow increasingly desperate to avoid the federal government's plan to resettle them in Papua New Guinea.

As many as 700 men are refusing food, and some are also refusing water, and say they fear for their lives if they are moved from the centre to temporary accommodation on the outskirts of Lorengau, the major town on the island.

The action escalated at the weekend after hardline comments made by new Immigration Minister Peter Dutton on Friday that the Manus Island detainees would "never arrive in Australia".

The protest has been building during the past week, with participants shouting "Freedom" and whistling and chanting.

Late last week, refugee advocates in contact with detainees estimated the protest involved more than half the 1035 detainees.