Firefighters try to put out the fire in an oil tank in the port of Es Sider, in Ras Lanuf, Libya, after an ISIS attack. Reuters Libya has become ISIS' new frontier as it faces territorial loses in its stronghold in Iraq and Syria, but it faces major roadblocks to establishing a true back-up capital there, according to a new report.

The Combating Terrorism Center report, published Thursday, said the terrorist group ISIS suffered "setbacks" in Libya and is struggling to expand there.

The West Point center noted that there's no doubt that ISIS will remain a "violent threat" in Libya as it's targeted in Iraq and Syria.

But the center also said ISIS' Libya base would be a "poorer and more constrained organization deprived of personnel, revenue, and the fundamental narrative tropes of governance and sectarianism that is has used to 'remain and expand.'"

US officials have reacted with alarm at how much ISIS (also known as the Islamic State, ISIL, or Daesh) has grown in Libya over the past year. And as the US-led anti-ISIS coalition continues hammering the group with airstrikes in Iraq and Syria, ISIS has directed foreign fighters to travel to Libya instead of the group's main base in Raqqa, Syria.

The effort paid off for ISIS in some ways. It now has an estimated 5,000 to 6,000 fighters in Libya, and it has been able to seize territory, establish "media points" to distribute its propaganda, and set up Sharia courts to enforce its strict version of Islamic law.

But ISIS' numbers in Libya still pale in comparison to their ranks in Iraq and Syria, where the group is estimated to have about 18,000 fighters, the report pointed out. Libya is three times the size of Iraq and Syria combined, but much of the country is desert.

And while ISIS has declared three "provinces" in Libya, some of them are its territories in name only. The Combating Terrorism Center noted that ISIS only really has control over areas around Sirte, a coastal city.

ISIS was able to capture Sirte with relative ease because it wasn't highly contested by other militias. Some of those rival groups took over parts of Libya in the chaos that followed longtime autocrat Muammar Gaddafi’s death in 2011.

Libya's landscape left "little room" for ISIS, the report stated. The country is mostly wide-open deserts that offer fighters little protection and cities in which many militias are fighting for control.

Libya Dawn fighters look at Islamic State militant positions near Sirte. Thomson Reuters

"When the Islamic State has tried to expand beyond Sirte to the east and to the west, it has run into towns that are controlled by other militias that rebuff them, such as Abu Qrayn in the west and Ajdabiya in the east," the report noted.

It continued:

In a recent interview, the Islamic State's new leader in Libya, identified as Abdul Qadr al-Naajdi, acknowledged as much, saying that "the number of factions [in Libya] and their disputes" had prevented the Islamic State from expanding its control beyond Sirte.

ISIS also might be running into financial problems in Libya. While the country has oil resources, which have been profitable for ISIS in Iraq and Syria, the group's volume of oil sales in Libya has been low, according to the center. But the largest share of ISIS' revenue in Iraq and Syria comes from taxing the local population and selling antiquities — not oil.

Additionally, the UN noted last year that ISIS wasn't yet organized enough in Libya to generate the revenue to sustain itself independently there. It's unclear how long ISIS could sustain its Libyan franchise if its bases in Syria and Iraq collapsed.

The Combatting Terrorism Center wasn't alone in its assessment of ISIS' strength in Libya.

Hassan Hassan, an expert on ISIS and resident fellow at the Tahrir Institute for Middle East Policy, wrote last month that ISIS is "still a minor player among the constellations of armed groups" in Libya and "can be defeated if the few factions committed to fighting it are supplied with the necessary aid."

Still, ISIS' expansion in Libya has been alarming, and the country's civil war and lack of a coherent government structure provide a fertile ground for extremism.

Europe is particularly concerned about ISIS's growth in Libya because of the country's proximity to the continent. Sirte is only 400 miles away from Sicily and other parts of Libya are close to Greece. US officials have said that ISIS now has the capability to organize attacks on Western targets out of its base in Libya.

Accordingly, Western countries might be drawn into fighting ISIS in Libya.

"I suspect that the Europeans are going to push more and more from the US to take action inside of Libya, so it's possible that we could see more airstrikes as well as possibilities of more special forces type of raids," Aaron Zelin, a fellow at the Washington Institute for Near East Policy, said in a podcast this week.

Zelin further said he doesn't think there will be some "grand invasion" of Western forces into Libya and that local forces need to be the ones to drive out the extremists.

Local militias have seemed willing to fight ISIS in other parts of Syria. And ISIS could have trouble drumming up popular support, as The Wall Street Journal reported last year that ISIS was having a hard time meeting the basic needs of the population. Gas stations and hospitals aren't functioning, and checkpoints make travel difficult.

As a civil engineer who recently fled Libya told The Journal: "Sirte has gone dark."