The peak bodies representing Australia's overseas aid and welfare communities are calling for the immediate release of all children from immigration detention.

The Australian Council for International Development (ACFID) and the Australian Council of Social Services (ACOSS) also said allegations of abuse in offshore centres should be referred to the Royal Commission into Institutional Responses to Child Sexual Abuse.

Space to play or pause, M to mute, left and right arrows to seek, up and down arrows for volume. Listen Duration: 3 minutes 29 seconds 3 m 29 s Listen to Naomi Woodley's report Download 6.4 MB

"I know that the Government and the Opposition say that we need to maintain the detention of children because we don't want to see drownings at sea, but to me that's a morally confused argument," ACFID executive director Marc Purcell told the ABC's AM program.

"It's never right to harm children. To stop one bad thing happening, you should not do another bad thing."

Mr Purcell said ACFID had intervened in the debate because its member organisations work on the "frontline" in places including Afghanistan, Syria, Sri Lanka and Myanmar with refugees who often end up in Australian immigration detention.

There are currently 97 children in immigration detention in Australia and 92 on Nauru.

Of the children in Australia, about 80 are due to go back to Nauru and, of the remaining 17, the Immigration Minister says most have a family member who will not be released for security reasons.

"In some circumstances, that mother, those parents have taken the decision that the mother and children can be released, and in some circumstances they've decided to remain in detention with the father," Immigration Minister Peter Dutton told Parliament.

He said there were 2,000 children in detention when the Coalition came to power in 2013.

"I want to work that number down to zero, but we have to provide a compassionate system to provide support, and we will do that," Mr Dutton said.

"The most important aspect is to make sure we don't allow the boats to restart, because I don't want detention centres to re-open as they did during Labor's time in Government."

But Mr Purcell says "the ends do not justify the means".

"The Government can look at how it wants to interdict the coast, what we're saying is that it is not justifiable to be putting kids in detention or in places such as Nauru where they experience significant mental harm and are vulnerable to abuse."

He said ACFID and ACOSS believed the Royal Commission into child sex abuse in institutions had the jurisdiction to examine events in immigration detention both in Australia and overseas.

"We don't believe that there is any legal difficulty in looking at the Commonwealth's actions in regards to how it responded over time, from the point where the regional detention centre on Nauru was established," Mr Purcell said.

The Greens are also planning a bill to put a time limit on the detention of children in immigration facilities.

The Greens' Immigration spokeswoman Sarah Hanson-Young will give notice today of a bill to set a 30 day time-limit on the detention of minors.