Newborn babies and their families are being secretly moved in the dead of night to Christmas Island detention centre, which is widely considered as unsuitable for young children by medical practitioners.

It is believed at least five two-month-old babies, their siblings and parents were given no notice as they were forced to leave Adelaide's Inverbrackie detention centre at 3am last week, without access to any legal advice.

On Monday, Immigration Minister Scott Morrison refuted this, saying the families were notified "well in advance of the 20 minutes that advocates are suggesting", but would not say how much notice was given.

The sudden relocations come as Immigration Minister Scott Morrison announced a $2.6 million educational package for school-aged children in the island's detention centre, run by Catholic Education Office of Western Australia, suggesting that the government is planning to increase the population of child detainees on the island.

Jacob Varghese, who is representing 26 Australian-born asylum seeker babies, said the families were living in daily fear that they would be "shipped off" to Christmas Island.