The decayed remains of at least one baby have allegedly been found buried in a coffin-like box at the headquarters of secretive religious sect Twelve Tribes.

Two police raids at Peppercorn Creek Farm in Picton and at a 78.5-hectare property near Bigga, south-west of Sydney, began on Monday.

The operation is part of an ongoing investigation into the high number of stillbirths within the community which rejects modern medicine and technology.

Excavation of the property was suspended after the alleged find on Wednesday afternoon, according to the Daily Telegraph.

Sources told the publication it was too early to tell whether the remains were of one or two infants, and the delicate retrieval process was delayed by heavy downpour.

Police have allegedly uncovered the remains of at least one infant at the headquarters of religious sect Twelve Tribes in Bigga, New South Wales (pictured)

New South Wales Police will not yet publicly comment on the operation, which was suspended on Thursday.

'There are no further updates in relation to Strike Force Nanegai at this time,' NSW Police said in a statement.

Three graves have been uncovered at the Bigga site - which has no running water or electricity and is only used when members of the secretive sect are exiled for questioning their beliefs, A Current Affair reported.

Police raids began at Peppercorn Creek Farm in Picton and a 78.5-hectare property near Bigga (pictured), southwest of Sydney, on Monday

Twelve Tribes is guarded about its privacy and members of the sect are expected to live by a set of rigid guidelines which govern almost every aspect of their daily lives

The raid comes after former member Rosemary Cruzado told the publication her late-term stillborn baby was buried at the Bigga property.

She believes her baby's death could have been avoided if she had seen a modern doctor earlier in her pregnancy.

Twelve Tribes has been investigated by police since September 2019.

The sect is a registered religious charity and has been in Australia since the '90s.

It has about 90 members in its Picton, Katoomba and Coledale communes and runs cafes in Sydney and the Blue Mountains.

The sect does not align itself with any denomination.

Members believe the Messiah will return if the church is restored to its original form in the Book of Acts - the first book in the new testament of the Bible.

Twelve Tribes has been guarded about its privacy and members are expected to live by a set of rigid guidelines which govern almost every aspect of their lives.

Communication with the outside world is largely forbidden and women are expected to be subservient to men and everyone must marry within the group.