Tension had been growing for months about Libya’s al-Nasr migrant detention centre in Zawiya, filled to bursting with those captured while trying to embark to Europe from nearby beaches.

Earlier this month the tension boiled over. More than 200 inmates staged a mass breakout and, in the panic, guards opened fire, scything down escapees with machine guns. When the smoke cleared, four migrants were dead and 20 more wounded, along with one guard.

“The situation at detention centres is horrific. Migrants are left to their own devices,” says Mark Micallef, of Migrant Report, which first broke news of the killings. He visited a nearby centre in December and says he could feel the tension: “I had never been to a detention centre that was so crowded.”

The killings highlight the crisis facing a country already overcrowded with migrants, as it faces a likely massive new influx after the closing of the Aegean migration route from Turkey to Greece.

Last year 87% of the 900,000 migrants making the journey to Europe came through Greece but, following the European Union’s new deal with Turkey, smugglers’ gangs are already sizing up Libya – which is mired in the chaos of civil war – as an alternative route.

“The sailing season has begun, the crossings [from Libya] in March were three times the figure of last year,” says Leo Dobbs of the Office of the UN high commissioner for refugees (UNHCR). “With the door closing on the Aegean, it’s quite possible that numbers will rise.”

The UNHCR says that Libya already has at least 100,000 migrants, who are packed into towns and cities along its western coast. The main obstacle to crossing the sea to Europe has been the weather, but the end of the winter storms has seen smugglers gearing up for a busy spring.

Patrols rescue people from overcrowded boats but critics say smugglers are using them as a ‘ferry service’. Photograph: AFP/Getty Images

Most of Libya’s current migrants are not Syrian or Iraqi refugees, who until now have been heading through Turkey, but people fleeing poverty from all corners of Africa.

The mayhem at Zawiya came, ironically, because of Libya’s own efforts to stem the tide. In recent months the coastguard has been catching ever larger numbers of those trying to reach Europe. The problem is they are then herded into 18 detention centres under the control of an interior ministry fractured by a 20-month civil war.

“Thousands of migrants and refugees are arrested and detained in Libya for migration-related offences. They face torture and other ill treatment,” the UN special mission to Libya reported last week.

A year ago, Europe promised firm action on Libya after 800 migrants were drowned when their overcrowded boat sank off Italy. The result was Operation Sophia, with a dozen naval craft since patrolling off Libya’s coast. The operation has seen a dramatic increase in rescues, while critics say the smugglers are using this naval force as a “ferry service”.

Sophia officials say success in intercepting and impounding 67 smuggler boats has been undermined because the gangs simply import fresh craft from Europe. Late last year Maltese customs found 20 semi-rigid boats aboard a ship destined for Libya, but took no action as they lacked the powers to seize the consignment.

“The patrols have been effective in saving lives – the central Mediterranean is far less deadly than it was this time last year,” says Micallef. “[But] Sophia doing intelligence work offshore is a complete waste of money. You need sources on the ground; smugglers aren’t stupid. We need a committed response.”

But both the EU and Libya’s fragmented authorities are dealing with smuggler networks that reach across the continent. A group of more than 100 Senagalese migrants, intercepted by the Tripoli coastguard, explained that each had been funded by a collection in his home village, on the understanding that, once in Europe, he would send money home to allow others to make the dangerous trip.

Other nations are more sophisticated still. Eritreans have agents in Tripoli who arrange accommodation for new arrivals, then find boats and skippers. With each migrant paying a minimum $1,000 (£708) for a passage, Sophia officials say smuggling gangs – many rooted in Libya’s all-powerful militias – earn between €250m to €300m a year.

Libyan Red Crescent workers recover a lifeless body of a drowned migrant on the beach in the eastern city of Tripoli, Libya. Photograph: Mohame Ben Khalifa/AP

Meanwhile, there is resentment among Libyans who are pushing back against the trade. The country has more than £100bn in overseas assets, but governmental chaos means its underfunded coastguard is putting to sea with too few boats and no effective radar.

In Libya’s westernmost town, Zuwara, anger exploded last year when 183 migrant bodies washed up on the beach. Protests saw the smugglers either close down or shift business to the nearby towns Sabratha and Zawiya, leaving would-be migrants stranded.

“The town is full, we have thousands of people here, what are we supposed to do with them?” says one Zuwaran activist, who didn’t want to be named.

An added problem is that the Islamic State is using Libya’s migrant flow as cover to bring in foreign recruits. Those volunteers blend with migrants going across the Sahara to Tripoli, then break off to head north-east to the Isis headquarters at Sirte, where the Pentagon says jihadi numbers have doubled to 6,000 in the past year.

As political pressure builds in Europe to stem the migrant flow, a more controversial option is now on the table, with the EU considering a plan for naval craft to go inshore, joining Libyan patrols to intercept migrants as they embark and return them to shore.

David Cameron raised the hackles of critics when he announced the idea at an EU summit last month, with some comparing it to Australia’s controversial interception policy. But a leaked Sophia report published by WikiLeaks shows that the plan, modestly titled Phase 2B, is now likely to clear its final obstacle, with Libya’s newly installed government of national accord expected to issue a formal invitation for the operation, removing legal obstacles.

Italian Sophia commander Rear Admiral Enrico Credendino reports that Phase 2B carries the risk of combat with heavily armed smugglers, writing: “We will be operating in a higher threat environment.”

EU leaders have yet to make a final decision on the operation, which would see naval ships operate in pairs, one to turn back migrant craft, the other ready to battle smugglers. But aid agencies fear migrants will be caught in the crossfire.

“It’s extremely worrying that conversations are still going ahead for the EU to illegally and forcibly push refugees back to Libya,” says Natalie Roberts, of Médecins Sans Frontières. “Pushing back desperate people to abandon them to violence and abuse in Libya is not an option that should ever be considered.”