The capital of Yemen is famous for its picturesque gingerbread houses, but in the last few days Sana'a has echoed to the sound of gunfire and explosions as President Ali Abdullah Saleh faces a last battle that could determine whether the country teeters into all-out anarchy.

Saleh survived what looked like a direct attempt to kill him when shells crashed into a mosque in his palace compound, reportedly leaving him lightly wounded but still defiant in the face of demands at home and abroad that he step down. The conflict looks more dangerous than ever now.

For nearly two weeks Yemenis have watched armed men crouching at makeshift barricades or firing from rooftops, the wounded bundled screaming into cars in a grim landscape of urban warfare. In Taiz, Yemen's industrial capital, police fired directly into crowds of unarmed demonstrators.

"In Change Square [Sana'a's tent encampment and homage to Cairo's Tahrir square] you can hear the bullets," Hamza al Shargabi, a surgeon and blogger, reported gloomily on Thursday. "The truce has failed. The fighting has become more and more fierce."

Yemen's version of the Arab spring is like the country itself — volatile, idiosyncratic and complex. Libya and Syria are racked by violence, too. But nowhere are events so hard to understand and the outcome more uncertain as in the land once known as "Arabia Felix", or happy Arabia.

Ironically, Saleh had been looking forward to 2011. In January parliament approved in principle a constitutional amendment that would have paved the way for him to extend his 32-year-rule indefinitely. In March, as protests swelled, he pledged not to stand again but still seemed determined to hang on.

Now, though, the former tank driver faces a final battle for survival. In the heart of the capital his bitterest rivals control key ministries, the ruling party HQ and one of the main police stations. The US, his uneasy partner in the war on terror, is ratcheting up the pressure – via a still hesitant Saudi Arabia next door – for Saleh to pack up and go into exile.

Initially the Obama administration refused to call on him to stand down, seeing him as a bulwark against al-Qaida, a view the Yemeni president was only to happy to encourage as it guaranteed him financing and support from Washington. But as Saleh's crackdown becomes ever more violent, that position has become untenable, even if it remains unclear what or who might follow him.

Yemen's many conflicts long pre-date the revolutions in Tunisia and Egypt, but they acted as a powerful catalyst – and a pretext for Saleh's enemies to act and to pursue old rivalries.

"Yemenis had been demanding change for 16 years since the civil war in 1994," said Muhammad al-Qutabi, a leading opposition activist and former senior government official. "Saleh agreed an agenda but he has reneged on it. The Arab spring encouraged Yemenis to come out to demand a better future."

If the protest movement has been slow-moving, its gains have been huge. Demonstrators have broken a wall of fear and given birth to a new way of life. Change Square started out with a bunch of rowdy students rattling the gates of Sana'a University to celebrate Hosni Mubarak's downfall. Now it is a fully-fledged shanty-town packed with poets, musicians, actors, art galleries, football tournaments and other portents of the nascent democracy so many of them are striving for.

The lines of dusty tents contain Yemenis from all walks of life: mutinous army officers, dissident tribal sheikhs, grey-haired socialists and young pro-democracy activists. In their midst are thousands of women, defying authority as well as the weight of tradition. In a country where most women are neither seen nor heard, the sight of 10,000 of them marching down a six-lane motorway in mid-April after Saleh accused them of "mingling with men" was too much for some to bear.

Tawakul Karman, the head of Women Journalists Without Chains, has become a much-admired figure agitating for press freedoms and staging sit-ins to demand the release of political prisoners. Her success in bringing so many protesters out, and keeping it largely peaceful, is a remarkable achievement

But this may be lost in the shadows of a looming war. On 23 May, after snubbing a third attempt at mediation by his Gulf neighbours, Saleh sent his forces to take on the leaders of Yemen's most powerful tribe, the Hashid, who have been bankrolling the opposition and supporting hundreds of thousands of protesters on the streets. It reinforced his reputation for ruthlessness – and for taking risks. The shootout between tribesmen and Saleh loyalists has left over 200 people dead.

Saleh's latest battle with the tribes, which he managed to juggle dexterously – and has famously compared to "dancing on the heads of snakes" – is part of a desperate attempt to prevent his downfall.

It is unclear who might replace him. The Gulf accord envisages elections within 60 days of a caretaker president taking over. But now, with the power struggle moving from negotiating table into the streets, the prospect of peaceful polls seems fanciful. It is just as likely that another strongman will push Saleh and his regime aside.

The man who may decide whether the violence fizzles out or flares is not the president but his half-brother, Ali al-Mohsen, a renegade general. Mohsen has been lurking on the sidelines, nervously keeping watch over the protesters in Change Square. He has called on his men to defy the president. "Beware of following this madman who is thirsty for more bloodshed," he said.

Elsewhere, swaths of Yemen are simply slipping out of Saleh's grasp. Zanjibar on the gulf of Aden is now under the control of 300 gunmen the government has described as al-Qaida militants – a sure way of engaging the attention of the US. Government forces have been pounding the city from the air but to no avail.

"Saleh is playing his last cards – to prove that only he can maintain stability," said journalist Abubakr al-Shamahi. "The question is whether people will buy that."

Away from the streets, there remains a significant silent majority who are too bogged down in the day-to-day struggle to exist to go out protesting. But with an economy marked by corruption, poverty and 40% unemployment, the uncommitted may have to take sides.

Yemeni and foreign experts alike warn that there is no quick or easy solution in sight. "If Saleh is removed from the equation tomorrow it still doesn't solve Yemen's problems," said Ginny Hill of the Chatham House thinktank. "It's not so much a fight between the government and the tribes as one between elite factions, though one of those does control a significant part of the instruments of the Yemeni state. We are not looking at a single transition but cycles of transition. Everything in Yemen is contested."

Abdulghani Iryani, a respected Sana'a analyst, is adamant: "Saleh's continuation in office will make civil war inevitable."

Yet the president, some argue, could be put under far heavier pressure from the west and the Saudis to sign and implement the Gulf deal. Sanctions, assets freezes and the sort of measures applied to Libya have not even been tried. "Saleh is unpredictable, devious and untrustworthy," said al-Qutabi. "But if he feels people breathing down his neck he will sign."

In numbers

17.9

Estimated median age in the country (by far the youngest population in the Middle East, by this measure)

23.6m

The population (2009) of Yemen, which has one of the world's highest birth rates, is increasing by 700,000 a year

64.9

Life expectancy at birth. Healthcare services are scarce in rural areas; only 25% of which are covered