On Monday, when the UN envoy Jamal Benomar and Abdullatif al-Zayani of the Gulf Co-operation Council flew into Yemen, they hoped for agreement on a transition plan that would see President Ali Abdullah Saleh step down after 33 years in power. Instead they found themselves in the middle of an urban battlefield, trying to negotiate an immediate ceasefire between rival elite families.

Negotiations over the timing of elections have gone back and forth between the ruling party and the opposition coalition since the spring. The beleaguered Gulf-backed transition plan still forms the basis of these negotiations, with support from the UN. But it increasingly appears that too much energy has been expended on technical and constitutional issues and too much faith placed in politicians from Yemen's formal institutions. Mediators have yet to find a successful way to manage the rivalries between unaccountable elite families who have slowly divided the capital, Sana'a, into zones of personal influence.

Those rivalries are playing a crucial part in the latest violence, which pits troops under the control of Saleh's family against those loyal to Ali Mohsen, the general who defected to the opposition and is related to Saleh through Saleh's mother's second marriage. The two men have shared power for decades but in recent years, as Saleh began to position his son Ahmed for succession, relations have grown increasingly tense.

When General Mohsen broke ranks after a sniper attack on protesters in March, many Yemenis feared that war between the two was imminent. Instead, they met one night at the house of the vice-president, Abd-Rabbu Mansour Hadi, and both agreed to step down and leave the country. But the deal fell apart and a protest camp sprang up, sprawling through the streets around the university under General Mohsen's protection. Saleh's rivalry with a powerful tribal family, the al-Ahmars, eventually led to fighting between Saleh's troops and tribal militia in Sana'a in May – ending when Saleh was evacuated to Riyadh for treatment after a bomb blast in his palace mosque.

Saleh remains in Riyadh, and the US state department continues to call for a "peaceful and orderly transition". But the Pentagon has been slow to cut ties to Saleh's son and his three cousins, who control Yemen's elite security and intelligence units. These units were created, funded and trained with western help after al-Qaida's 2000 attack on the USS Cole , and this local "capacity building" has formed a central pillar of the west's anti-terror strategy in Yemen's terrorist threat for much of the last decade. John Brennan, the White House counterterrorism adviser, stated this month that "counterterrorism co-operation with Yemen is better than it's been in years".

But the Pentagon's reliance on Ahmed and his cousins is distorting Yemen's domestic politics, and the options for transition. Even if Saleh himself might be ready to stand down – which many doubt – it is clear that he still expects a prominent role for Ahmed.

Meanwhile, Ahmed and his cousins are entrenched in the presidential palace in Sana'a, and gunmen under their control opened fire on demonstrators on Sunday. Neither Ali Mohsen nor the al-Ahmar family show any sign of consenting to a transition deal that leaves Saleh's inheritors in place.

Yemen's protesters are bearing the bloody brunt of these elite rivalries. Not all the protesters are independent – many are allied to a party funded by one of the al-Ahmar brothers. But restructuring the military and purging the security services of corruption are among the independent protesters' top priorities. They want to see an end to the military, political and economic control of Saleh and his relatives. But they are paying for this demand with their lives.