The fall from power of Tunisian president Zine al-Abidine ben Ali is one of those widely unpredicted turns of events that hindsight quickly labels inevitable.

Corrupt authoritarian regimes are generally brittle and Mr Ben Ali's was no exception. But few anticipated how quickly a spate of angry demonstrations could become a regime-changing rebellion. Other governments across the region, with populations hardly less repressed than Tunisia's, will look on in fear.

Mr Ben Ali was considered by western diplomats to be a relatively reliable fixture. Under his 23-year rule, the country had the status of a minor player in North Africa – avoiding involvement in wider Middle East disputes and carving out an economic niche as a Mediterranean holiday destination.

Meanwhile, the president, his wife and their extended family built a lucrative commercial empire. Political dissent has been crushed and media stifled. In a dispatch sent in July 2009 – one of the secret cables published earlier this year by WikiLeaks – the US ambassador to Tunis described rising frustration among ordinary Tunisians as a result of "First Family corruption, high unemployment and regional inequities". He also noted that major change would "have to wait for Ben Ali's departure".

Tunisians clearly shared that view.

The trigger was the suicide of Mohamed Bouazizi, an unemployed student who set fire to himself in protest after police confiscated the vegetable stall that was his living. A wave of sympathetic protest then grew, becoming ever more determined in response to brutal attempts at suppression by police.

While the mass mobilisation was sudden, the frustration it expressed has been a generation in the making. This revolution is demographic as well as economic and political. One in five Tunisians is aged 15-24 (as compared with around 1 in 10 in Britain) and youth unemployment is at least 30%. Joblessness is particularly high among university graduates. That is a phenomenon common to many Arab countries as a growing graduate population combines with a decline in the state's ability to provide public sector jobs, while private sectors remain underdeveloped.

The result is a huge cohort of young people with too much time and not enough money. In Algeria, they are known as the "hittists", meaning the people who lean against walls – an emblem of bored, disaffected youth. Members of this generation also have ways of sharing information online that, while sometimes disrupted by state censorship, cannot be entirely silenced. They are potentially a vast source of political upheaval.

That is one reason why governments from Morocco and Algeria along the Mediterranean coast to Egypt, Jordan, Syria and the Gulf will be watching Tunisia with alarm. These are diverse states, but with common features: ossified politics and corrupt elites, lacking any governing principle other than the urge to resist demands for change from liberals and Islamists. They also suffer from cultural and academic sterility – the suffocation of free thought that might seed political and social renewal.

Stability for such regimes relies on a combination of state force and public apathy. It is the latter that changed so markedly in Tunisia. Especially worrying for other Arab leaders will be the fearlessness of the crowd, prepared to confront riot police firing live rounds. Authoritarian regimes rarely survive for long once the illusion of invincibility is shattered.

In recent weeks, there have been angry protests over rising food prices and unemployment in Jordan and Algeria. There were riots in Egypt last November after disputed parliamentary elections.

The lack of legitimacy does not mean Arab regimes are about to topple. The experience of communist states in eastern Europe in the 1980s shows that bankrupt systems can cling on in protracted, decaying endgames. But ultimately, they do fall.

The comparison is revealing. During the cold war, western powers routinely supported the aspirations of captive citizens against their rulers. The west also cultivated dissident intellectuals, recognising the moral power that flows from the defence of open minds against closed systems. By contrast, the US and Europe have propped up blinkered, failing Arab regimes, judging them to be bulwarks against Islamist radicalism. It is a terribly misguided strategy, not least because it conforms to the jihadi narrative of a west hostile to the interests of ordinary Muslims.

In Tunisia, the opposition is not especially Islamic. Mr Ben Ali's attempts to label the demonstrators "terrorists" in the early days of the uprising was a sign of desperation. Presumably, he hoped to buy US sympathy. Much past American policy in the region gave him grounds to think such a tactic might work.

But there are signs of a more sophisticated approach coming from Washington. Last week, secretary of state Hillary Clinton spoke damningly of the failure by Arab states to modernise. Spreading opportunity to ordinary people, she said, was the surest guarantee against extremism.

Europe has been slower to speak out on behalf of disenfranchised Arabs. France, the former colonial power in Tunisia, supported Mr Ben Ali until the very last moment. One minster offered to send riot police to help shore up the regime.

The EU has an obvious interest in fostering political and economic regeneration on its Mediterranean border. The goal has often been discussed at regional summits, but progress is never made because modernisation means breaking up the power monopolies of corrupt elites. It takes a concerted effort of diplomatic and commercial power to encourage such regimes to change. The alternative, as has been proved in Tunisia, is violent change forced from below.

It is unclear whether the country will emerge from this tumult with better leaders. There is at least potential for progress without Mr Ben Ali. That is a warning to leaders across the region. But it also contains a lesson for Europe and the US. Presidents-for-life offering bogus protection against phantom terrorists are not reliable friends. The surest allies for the long term are the ordinary people in Arab countries whose aspirations are being systematically thwarted. It is their friendship the west must be conspicuously courting.