Thousands of people have joined a Palm Sunday rally in Sydney, calling on the Federal Government not to send almost 270 asylum seekers back to Nauru.

It comes after the High Court last month ruled that Australia's offshore detention at Nauru and Manus Island is legal.

The judgement cleared the way for 267 asylum seekers brought to Australia for medical treatment to be returned to Nauru.

Refugee advocates said many have been moved into community detention for now but the Government is threatening to ultimately deport them back to Nauru.

Sydney paediatrician Doctor David Isaacs told the rally he was sent to Nauru to treat sick children in detention in 2014, and found conditions there appalling.

"I was horrified by what I witnessed there," he told the rally in Sydney's Belmore Park.

"This is not a question of human rights, it's a question of human decency.

"If you were fleeing from oppression and in fear of your life and your children's lives, what would you want done to you? How would you want to be treated? Who will speak out for these people hidden away there?"

He accused the Government of using the threat of imprisonment under the Border Force Act to try to silence him and others from speaking out.

Church leaders from various faiths also addressed the rally, reiterating their offer of sanctuary to asylum seekers at risk of being deported.

Father Shenouda Mansour from the Coptic Orthodox Church also called for Nauru and Manus Island detention centres to be shut down.

"It's time to say we do not want any children in detention centres," he told the rally.

"We want them all released from these detention centres into the community."