Killing of US ambassador Chris Stevens drives away foreigners and crushes hopes of locals who fear rise of extremism

Night-time in Benghazi is not for the faint-hearted. The rattle of gunfire shatters the small-hours silence outside the Tibesti hotel, where bigwigs and the city's few remaining foreigners are holed up.

Outside, guards from a mix of police and militia units scatter to firing positions. Jeeps with red and blue flashing lights arrive with reinforcements as an orange muzzle-flash lights up the waste ground between the hotel and the sea lagoon.

Then it stops. The gunmen may have been local drunks, or bored militiamen, or jihadists; nobody knows.

With Libya's government in chaos and extremists on the loose, the tension of Tibesti's guards is understandable. The killing of the US ambassador, Chris Stevens, and three diplomats in the consulate compound south of the city centre has left Libya staring catastrophe in the face.

A year ago the proud boast of this eastern port city was that it was the cradle of last year's Arab spring revolution. Today its population fears being known as the place where jihadists killed the first US ambassador to be assassinated since 1979.

"Everything we have been working for has been crushed," says Hana el-Galal, one of the city's most prominent civil rights activists.

Until last week's attack she had been confident after the success of trouble-free elections in July. She and other rights groups had been due to meet Stevens the day after he died. "We lost a friend and nationally we lost a lot."

Signs of just how much Libya threatens to lose if the killers are not brought to justice and the militias disciplined are all around: all foreign missions in the city have been evacuated, together with the United Nations.

A year ago French flags were sold in the central Courthouse Square, in celebration of the French air strikes that saved Benghazi from Muammar Gaddafi's tanks. This week, French journalists fled the city, fearing reprisals from the publication of cartoons depicting the prophet Muhammad by a Paris magazine.

There are no French flags for sale now. "We want the foreigners to come back," says an electrician trained at Marconi in Chelmsford. "The extremists, they put this anger in the minds of the people." He does not give his name.

The Venezia cafe, Benghazi's plushest nightspot, sits across from the bullet-riddled back gate of the burned-out consulate compound. Since the attack the diplomats who used to eat there are gone. "They used to come every day," says Mahmoud Ged, an Egyptian waiter with a nervous smile. "I don't know when they will be back."

Across Libya, business conferences are being cancelled, a British trade delegation the latest to scrub a visit Libyans had hoped would kickstart commerce. Libya desperately needs foreign knowhow to rebuild the furniture of a modern state after 40 years of dictatorship. It may not come now.

Dr Mohamed Jamal, a physicist who returned to the city from working in Poland, insists Benghazi is not a jihadist city. Like many here, he says that though Libyans were deeply upset by the infamous film insulting Islam, there was no anti-American protest last week, simply a well-planned attack on the consulate. Now he fears all Libyans will be regarded as extremists. "The main problem between religions is communication," he says.

Last week's attack was not an isolated incident. Recent months have seen extremists vandalise Benghazi's Commonwealth war graves and strike five diplomatic targets, among them the convoy of the British ambassador. Add in 14 assassinations of former Gaddafi-era officials in as many weeks and you have a city teetering on the edge of anarchy. Blame, across this city, is fixed on elements of the Islamist Ansar al-Sharia brigade.

Libya has a new parliament but not yet a cabinet. The existing cabinet, inherited from the previous transitional government, has been conspicuous by its absence: its prime minister, Abdurrahim el-Keib, has not appeared in Benghazi, and nor have key ministers. "We feel abandoned," says Galal. "There is a problem of weak government, they leave a vacuum, when your prime minister does not come to visit you."

The abandonment is felt as keenly regarding the United States. Nobody in this city believes the conclusion of the US ambassador to the UN, Susan Rice, that the attack on the consulate grew out of a demonstration. Cynics think the Americans want to minimise the killings.

"What demonstration?" says a Libyan-American who watched the attack from the Venezia. "There was no demonstration. They came with machine guns, with rockets." He also will not give his name.

The extremists, whoever they are, are people nobody in authority wants to tackle. Benghazi's police, gendarmerie, interior ministry and justice department play pass-the-parcel over the lack of any serious investigation. Even mention of Ansar al-Sharia gets a reaction akin to Harry Potter mentioning Lord Voldemort – the very name makes officials cringe.

Benghazi's chief prosecutor, Saleh Adem Muhammad, is unhappy to be landed with the job of chief investigator. "I'm not frightened of Ansar al-Sharia," he insists. "Ansar Sharia is not under the interior ministry or the defence ministry so for us they are just civilians. It is an illegitimate brigade."

So will he order it to disarm? "This subject has nothing to do with my job."

The one politician who has made the killing his business is the president of Libya's parliament, the de facto head of state, 72-year-old Muhammad Magariaf. He led exile groups in Britain and the US for 30 years, and is a lone voice at the top blaming elements of Ansar al-Sharia for the killing and linking them to al-Qaida in the Maghreb. Meeting the Guardian, he had tears in his eyes as he said this was the "turning point" for Libya.

Magariaf wants to close down Ansar al-Sharia but says he is shorn of support by government forces who stood aside last month to allow Salafists to bulldoze a Sufi shrine in Tripoli. Instead, he has turned to the rebel militias of last year who have formed the National Shield, a parallel army to the existing government army.

Benghazi's garrison commander, General Hamad Belkhair, says he is ready to help. At a late-night meeting, amid tight security in a location in Benghazi's back streets away from his headquarters, he says: "We will do what the government asks, even going to (disarm) this Ansar al-Sharia."

But first the government has to ask. Much now depends on a mass rally in this city on Friday, which is due to march on the headquarters of Ansar al-Sharia, demanding they disband. If it is big enough, organisers hope people power will trump the lack of governmental resolve.

Ansar al-Sharia insists it is not to blame for the attacks, pointing out that no evidence has been presented to the contrary. But things are tense around their compound. A guard punches the Guardian's translator in the stomach at a second visit, ordering us not to return.

What happens next depends on the size of Friday's rally. "We want Libya to be cosmopolitan, we want more foreigners to come here," says Eamon el-Badn, a champion jockey at a stables on the outskirts of Benghazi, its crumbling brick stalls contrasting with well-kept mounts. "The guy who did that film, we know he does not represent Americans, and these extremists, they do not represent Islam."