Eleven children who were rescued from a compound in New Mexico on Friday where they were being held by their 'Muslim extremist' parents were discovered after a message was somehow sent to the outside pleading for help.

The children are aged between one and 15. Some are siblings and some are cousins.

They had been held underground at a ramshackle compound in Alamia, New Mexico, for months.

The FBI was monitoring the site for two months but never searched it because they did not have an arrest warrant, Taos County Sheriff Jerry Hogrefe said on Monday.

On Friday, however, the sheriff's office intercepted a message that was written either by one of the children or by one of their mothers which read: 'We are starving and we need food and water.'

Authorities will not say how the message was delivered or who gave it to them but Sheriff Hogrefe said it meant he could not wait any longer for federal authorities to move in on the site.

Sisters Subhanah Wahhaj, 35, (left) and Hujrah Wahhaj, 38, (right) were arrested on Sunday

Jany Leveille, who also goes by the name Maryam, was also arrested for child abuse on Sunday

Siraj Ibn Wahhaj, 39, had an AR-15 rifle, 150 rounds of ammunition and four loaded pistols at the site. He has been on the run since December when he vanished with his three-year-old disabled son from Georgia. The toddler was not found. His brother-in-law Lucas Morton (right) was also charged with harboring a fugitive. He is married to Siraj's sister Subhanah

He enlisted a special forces team to lead the raid because he knew the men inside were heavily armed, he added.

The children were being held there by 39-year-old Siraj Ibn Wahhaj, an imam's son from New York City who has been missing since December when he abducted his three-year-old special needs son from Georgia.

It was this abduction which triggered the FBI investigation.

On Friday, Wahhaj was found in the compound with an AR-15 rifle, five magazines loaded with 30 rounds and four loaded pistols.

This is the New Mexico compound where 11 children were found on Friday being held by their Muslim extremist father and uncle, Siraj Ibn Wahhaj, along with their parents

Also on the compound was his brother-in-law, Lucas Morton and three women; Subhanah Wahhaj, 35, Hujrah Wahhaj, 38, and Jany Leveille, 35, who is also known as Maryam.

Sunbhanah and Hujrah are Siraj's sisters and Jany is his wife. She worked for his father, Imam Siraj Wahhaj, at the Masjid At-Taqwa mosque in Brooklyn before relocating with her husband to Georgia in 2014. She is not AG's mother.

All three women were charged with child abuse on Sunday after initially being released by police who said they were 'brainwashed and intimidated' by the men.

Siraj's three-year-old son AG, who has hypoxic ischemic encephalopathy and requires daily medication, has been missing since December and was not at the compound

Not found was Wahhaj's three-year-old son AG who cannot walk or talk due to hypoxic ischemic encephalopathy, a condition caused by brain damage.

When police closed in on the property on Friday, his father initially refused to follow orders.

Police worried that the raid would end violently because of the fact he was so heavily armed.

'We had learned the occupants were most likely heavily armed and considered extremist of the Muslim belief.

'We also knew from the layout of the compound they would have an advantage if we didn’t deploy tactfully and quickly,' Sheriff Hogrefe of the Taos County Sheriff Department said in a statement.

The children had been at the site for months without food and were so skinny their ribs were showing.

The only food found on the compound was a box of rice and some potatoes.

Sheriff Hogrefe described the conditions as the saddest he had ever seen in 30 years of police work.

'We all gave the kids our water and what snacks we had – it was the saddest living conditions and poverty I have seen,' he said, adding that the children looked like 'third world refugees.'

The conditions inside the compound were described by police as the worst they had seen in 30 years

There was no food at the compound and the children had terrible personal hygiene, police said

Amalia is 145 miles northeast of Albuquerque and in an isolated high-desert area near the New Mexico-Colorado border

'I've been a cop for 30 years. I've never seen anything like this. Unbelievable.

'They were skinny, their ribs showed, they were in very poor hygiene and very scared.

'They had 'no shoes, personal hygiene and basically dirty rags for clothing,' he told ABC.

Siraj Wahhaj has been on the run since December when he vanished with AG after telling his mother he was taking him to the park.

AG's mother Hakima Ramzi has made desperate social media appeals to find her son again.

Her friends have also set up a GoFundMe page to help pay for the search.

The family has been missing since January when their father, Imam Siraj Wahhaj, appealed for information about them on Facebook

The children's grandfather is Imam Siraj Wahhaj, the leader of the Muslim Alliance of North America and the leader of a mosque in Brooklyn, New York City

The toddler suffers from seizures, developmental delay, wears braces on his legs, cannot walk, and requires daily medication.

His parents were married for 15 years until he vanished in December.

In a Facebook plea for information about her son and husband's whereabouts, Ramzi appealed directly to Subhannah, one of the three mothers found on the compound.

She said: 'It's not about you.

AG's mother Hakima Ramzi was married to his father without incident for 15 years, she said. She has been pleading for AG's return since December and is seen above in a Facebook video appealing for help to find them

'This is all about my son. He needs his medication. My worry is not about Subhannah and her kids. My worry is about my son because he is sick.

'He needs his medication, he needs everything. I don't know if he is alive. Please, please, please, I need your help.

'I need my son back to me and then I will leave you alone,' she said.

Wahhaj is the son of Imam Surraj Wahhaj, 68, who belongs to the Masjid At Taqwa mosque in Brooklyn, New York.

The earliest record of him in the United States dates back to 1983.

It is not clear where he lived beforehand or if he was born in the US.

In January, his father pleaded for information about their whereabouts.

'Dear Brothers and Sisters, please make duas for the safe return of our children and grandchildren: Siraj, Hujrah, Subhanah Wahhaj, son in law Luqman (Lucas) Morton, and daughter in law Maryam (Jany) Leveille and their children (our 12 grandchildren).

'We believe they may be traveling together,' he said in a Facebook post.

He has not commented on his children's arrest. Imam Wahhaj is the leader of The Muslim Alliance in North America.

Police are still looking for three-year-old AG.