Since the uprising against Syrian President Bashar Assad started almost four years, the people of Lebanon have feared the conflict would creep across the 230-mile border they share with Syria, tipping a once war-ravaged nation back into internal bloodletting.

Now both the Islamic State of Iraq and Greater Syria (ISIS) and the Al-Qaeda-affiliated Nusra Front are occupying part of the Lebanon/Syria border and thousands of Lebanese men have gone into Syria to fight, many of them for the Shiite militia Hizballah.

The border pressure has raised tensions in Lebanon, which has endured bombings and suicide attacks against both Sunni and Shiite targets, but the country has not crumpled under the pressure.

When Hizballah acknowledged last year that its fighters were in Syria, many Lebanese accused the Shiite militia of dragging Lebanon into Syria’s war. This increased divisions between Hizballah and Sunni factions such as Saad Hariri’s Future Movement and the March 14 alliance. Tensions were rising as Lebanese factions, communities, parties and individuals found themselves aligned with opposing sides in Syria’s civil war, some supporting the Assad regime and others the Sunni rebels fighting against the Shiite-aligned President.

READ MORE: Inside ISIS, a TIME Special Report

But something unexpected has happened over the past year. The intensifying conflict and, in particular, the threat of ISIS has actually seen rival Lebanese factions move close together, says Yezid Sayigh, a Senior Associate at the Carnegie Middle East Center, a think tank in Beirut. “Here, I’m particularly thinking about Hizballah and other Shiite factions like Amal on the one side, and the Future Movement, March 14 camp and Saad Hariri on the other,” says Sayigh.

The rivals were galvanised by the capture and execution last year of several soldiers of the Lebanese army by ISIS and Jabhat al-Nusra. The brutality of the killings, which were filmed and disseminated on social media, persuaded moderate Sunnis, Christians and Shiites that they had an external enemy that was more dangerous than their respective domestic rivalries. “In Lebanon, the ISIS threat has been sort of crystallizing or focusing minds on the threat of ISIS and therefore creating an opportunity for the main political camps to converge on a few key policy measures,” says Sayigh. “There has been a sharpening of the lines, with more people moving into a position not so far from that of Hizballah.”

He cautions, however, that this convergence won’t trump parties’ larger political interests. “It’s a unifying factor, but not to the point where anyone is going to set aside their private agendas,” he says.

About one-quarter of Lebanon’s four million people are Sunni Muslims, like ISIS. But the group hasn’t been able to make the sort of inroads it did with disenfranchised Sunni populations in Iraq and with rebel factions in Syria.

“It’s seen somehow as a social alien as it comes from eastern areas of Syria and northern Iraq, tribal clan communities, which are alien to much of Lebanon,” says Sayigh. “ISIS doesn’t have a natural social base in Lebanon,”

One place ISIS has found a foothold in Lebanon is inside the country’s Roumieh Prison, just outside Beirut. Pictures published on social media show smiling Islamists in front of a large ISIS flag hanging from the bookshelf in the prison’s library.

In June, ISIS fighters appeared in a video posted on social media singing for the liberation of their “brothers” in Roumieh, in front of a black and white ISIS flag. “We want to crack the gate,” sing the men in the video clutching weapons, their faces covered by balaclavas.

They compare the prison to Abu Ghraib, the Iraqi prison that became known for the abuse of prisoners at the hands of American military and intelligence personnel. And like other American-run prisons such as Camp Bucca in Iraq, where it now seems much of ISIS’ leadership met, Roumieh may also be a breeding ground for extremism.

The prison’s Block B housed hundreds of accused militants, living in close quarters. In the crowded cells, young men accused of small crimes were held alongside those convicted of masterminding more serious attacks. Some who worked with the prisoners say many of the inmates were radicalized inside. It was from within the walls of Roumieh, officials believe, that pro-ISIS militants organized a double bombing in January that killed nine people and injured dozens more in the northern Lebanese city of Tripoli. After the attack, the prison authorities cleared Block B, moving the inmates to other wings, making radicalization more difficult and promising reforms to the prison system.

Sayigh says the prison re-organisation is an example of how Lebanon is responding to the threat of ISIS and other militants targeting the country. “Ironically, it’s sort of making the Lebanese state a bit more organized, responsible or proactive, which is not typical of Lebanon.”

Get The Brief. Sign up to receive the top stories you need to know right now. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Contact us at letters@time.com.