At least 5,600 men, women and children from 33 countries who traveled to live in ISIS' caliphate have now returned home, according to a new report.

The Soufan Center, a non-profit that examines global security issues, also suggested the inflow of foreign fighters to Iraq and Syria has come to a virtual standstill as ISIS continues to lose territory.

However, the report highlights that the question of what to do with so-called foreign fighters and their families has left authorities in their home countries puzzled.

"It's a hell of a problem," said Richard Barrett, a senior adviser at the Soufan Center who authored the report. "It's very resource-intensive to identify who these people are, track them down and then to do something about them."

While the report says returning fighters have not as yet added significantly to the threat of terrorism around the world, it warns that ISIS' leadership will likely be looking to supporters overseas, including returnees, "to keep the brand alive."

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

The report said: "If on return they begin again to feel as rootless and lacking in purpose as they did before they left, then they are unlikely to settle back easily into a ‘normal’ life, and as ISIS increases its external campaign, both through action and propaganda, returnees may be particularly vulnerable to contact from people who were part of the network that recruited them, or appeals for help from ex-comrades in arms."

But Barrett says the volume of foreign fighters returning should not be looked at from purely a numerical point of view.

"We have to be cautious of thinking that all [5,600] would go back and be a problem, but I think it's fair to assume that probably some will be," he said.

More than 40,000 foreign citizens from more than 110 countries have traveled to Syria and Iraq since ISIS declared its caliphate in 2014.

Related: What Should West Do With Fighters Returning From Syria & Iraq?

The Soufan Center's report quotes an estimate by the Radicalization Awareness Network that about 30 percent of the approximately 5,000 residents of the European Union thought to have gone to Syria and Iraq had returned home. In the United Kingdom, the number is believed to be closer to half.

The report estimates only about seven out of the around 129 fighters who left the United States to join ISIS are now back in the country.

It also identifies five categories of returnees, ranging from those who failed to integrate within ISIS to hardened terrorists sent elsewhere to fight for the caliphate.

The report points out a subset of returnees even harder for states to address — the women and children who joined ISIS, or in the case of some children, were born in the caliphate. It urges proper mental health and social support mechanisms for the children, some of whom will have been taught to use weapons and kill.

"It's one of the hardest things of all to rehabilitate children who have been recruited by violent extremist groups," Barrett said. "Once trained, they are incredibly committed and loyal."

Barrett says very young children get indoctrinated by ISIS and those older than 9 are often given military training, an emotional baggage that won't be easy to drop should they ever leave the caliphate.

"They will be a lot of mental health issues," he said. "And you can't just put them in a child detention center — you have to treat them. Some will be turned around quite easily and will follow the next 'mother figure' that comes along, but others will be unpredictable to say the least."