After some years of peace, the western Balkans are again descending into instability. Across the region, people are taking to the streets, demanding the resignation of governments. Thousands are fleeing abroad in search of jobs and opportunities. A violent strand of Wahhabism is taking hold among the region’s Muslim population. Perhaps most worryingly of all, the threat of disintegration is returning, as malcontent minorities try to divide their states.

Bosnia has long been the most dysfunctional state in the region, wasted by civil war in the 1990s and afflicted by ethnic divisions ever since. The Serbs and Croats have never abandoned their goal of separation. Milorad Dodik, the president of Republika Srpska (Bosnia’s Serbian “entity”), is being squeezed by political rivals at home and investigated by police in Sarajevo for alleged money laundering. To shore up his position, he has threatened a referendum on independence for Republika Srpska, scheduled for 2018.

Not far behind is Kosovo, an impoverished plateau in the Šar Mountains. It is unrecognised by half of the world, run by a corrupt elite and saddled with an embittered Serb minority. After years of resistance, Kosovo’s Serbs have recently extracted the right to territorial autonomy from the country’s notional EU supervisors. This has provoked a ferocious backlash from Albanian nationalists, who have attacked the parliament and held a series of violent street demonstrations.

Meanwhile, Macedonia is in chaos following the leaking of tapes that led to accusations that the former prime minister Nikola Gruevski had spied on the population and had been involved in corruption, electoral fraud and outright criminality. This has outraged the unhappy Albanian minority, which blames its leaders for upholding an illegitimate government instead of its community rights. In response, this group is demanding the federalisation of the state, auguring its potential disintegration. In the Balkans, it all eventually comes back to nationalism.

While local factors go some way to explaining the turmoil, however, they don’t tell us why the region as a whole is experiencing such instability, or why events are turning for the worse. The key to understanding the malaise is to recognise the Balkans’ position as a borderland between great powers. Throughout history, when one of these powers has wielded hegemony in the region, or a concert of powers has agreed a settled division, peace has generally prevailed. When no single power has been dominant or, worse, when powers have competed for control, chaos has invariably ensued. The Ottoman era marked the longest period of peace in modern times. But when the empire went into decline in the 19th century, nationalists across the Balkans seized the opportunity for independence – first the Greeks, then the Serbs and finally all the rest, helped by an opportunistic Russia, which sought to destabilise its Ottoman rival.

Violence continued into the 20th century as the collapse of the European land empires untethered the region. The Balkan wars of the 1910s, in which emerging states such as Albania, Montenegro and Serbia fought to define their borders, were followed by two world wars, in which Austria, Germany, Italy and the Soviet Union all invaded the territory.

The western Balkans were finally pacified in the postwar period. Bulgaria and Romania passed to Soviet control and the two superpowers agreed to maintain a unified Yugoslavia as a strategic buffer between their respective spheres of influence. Wedged between Nato and the Warsaw Pact, with no room for manoeuvre, and a strongman, Tito, to maintain order at home, the locals put their enmities to one side.

With the end of the Cold War, the superpowers largely lost interest in the Balkans and released their grip on Yugoslavia. Romania and Bulgaria, free of ethnic entanglements, managed to find their balance. But the western Balkans were set adrift and violence returned as Serbs and others took up arms to forge a new order on the wreckage of the old multinational communist state.

Stability was eventually restored when the United States, which emerged as the undisputed superpower in the 1990s, imposed a new imperial settlement on the region. In Croatia, Washington helped the local army to crush the breakaway Republic of Serbian Krajina. In Bosnia, the US bombed Serbian positions, decisively tipping the balance of power in favour of the central government that had endured three years of military losses. In doing so, Washington was interested in promoting not only peace but also justice. After the brutality of the Serbian military campaign, with its terrorising and expulsion of other ethnic groups, basic morality determined that the Serbs be denied their wartime goal of independence from the rest of Bosnia.

The result was the Dayton Agreement of 1995, a delicate compromise in which Serbs (and Croats) agreed to remain part of a unified Bosnian state. In return, the Serbs were given a self-governing entity – Republika Srpska – on half of the territory of Bosnia, while Croats gained limited self-government within a new Muslim-Croat federation.

Having dictated the terms of Dayton, the US, in effect, became its guarantor, supported by its European allies. It established a huge civilian presence on the ground, intended to steer Bosnia towards a durable peace. The Office of the High Representative adjudicated in ethnic disputes, clamped down on nationalist rhetoric and focused the locals on questions of social and economic reform rather than borders and territory. If politicians refused to co-operate, they were removed from their positions or presented with criminal charges. Nato troops on the ground did the enforcing.

***

When conflict broke out in Kosovo between Albanian separatists and their rulers in Belgrade in 1999, the US similarly imposed itself on the territory, using overwhelming force to expel the Serbian army, before setting up a civilian mission, Unmik, to steer a unified country towards a sustainable peace, as it had done in Bosnia.

With stability in both of these countries still fragile, nationalist conflicts elsewhere in the western Balkans could not be allowed to jeopardise Washington’s unfinished efforts at multi-ethnic state-building. When Macedonia’s unhappy Albanian minority launched a short-lived insurgency in 2001, the US clamped down on it with a settlement that forced Albanians to abandon the goal of separation in return for limited self-government. Macedonia held.

A similar logic applied to other states in the region. Through the 2000s, the US extended its presence in Albania, slowed down the secession of Montenegro and, with the fall of Slobodan Milosevic in 2000, implanted itself in Serbia, where it demanded democratic reform and Western integration in place of a discredited nationalism.

In this respect, the late 1990s and early 2000s can be seen as marking a restoration of order in the western Balkans after the chaos of the immediate post-Yugoslav period. With Washington at the helm, buttressed by European manpower and money, nationalists and separatists were disempowered and multi-ethnicity became the watchword. Many locals were frustrated with the American-led settlement, whether they were minorities such as Bosnian Serbs and Macedonian Albanians, who had ended up living in someone else’s state, or Bosniaks and Macedonians, who opposed the territorial concessions granted to violent minorities.

Confronted with overwhelming American power and the absence of any other power to whom they could appeal, there was little that the peoples of the western Balkans could do to change things. Turkey was content, concerned above all with peace on its land route to the markets of Europe. And Russia, while sympathetic to the plight of the Serbs, had no wish to encourage separatism in places such as Chechnya by questioning the new order in the Balkans.

However, this attempt at order was not to last. Matters went into reverse in the second half of the 2000s when the US withdrew its forces from the region to concentrate on more pressing issues elsewhere in the world. Its parting shot was to engineer the independence of Kosovo in 2008. With the last piece in the Balkan jigsaw in place – at least as Washington saw it – the US left it to the EU to finish the job of transforming the region’s turbulent states into prosperous and stable polities.

In tactical terms, the EU adopted a different approach to the US, replacing the hard power of the American military with the soft power of inducement – not least because, without an army, the EU had no real stick to wield. What it offered instead was a compact known as “conditionality”. For its part, Brussels agreed to admit the western Balkans into the EU, with all the benefits that this entailed – money, trade, freedom to travel and the chance for the locals to be reunited with their ethnic kin in a borderless Europe. And, for their part, the locals were expected to meet the conditions for entry to the EU, as the central Europeans had done before them.

Almost from the start, however, things failed to go to plan because the locals wouldn’t knuckle down to reform. By definition, the states of the western Balkans were eastern Europe’s laggards, blighted by the legacies of war as well as nostalgia for Yugoslav-style socialism and the absence of any tradition of democracy, liberalism or free markets.

Sometimes, the EU pushed issues that were important in a Western context, such as prison reform or gender rights, but just not a priority for the locals, who were more concerned with establishing the territory of the state, or changing the state they lived in. At other times, the required reforms cut across the interests of the elites who were making fortunes running a rentier economy.

***

The most resistant state was Bosnia, where the conflict never truly ended and where each ethnic group used the integration process to advance its core political goal: centralisation in the case of Bosniaks, separation in the case of the Serbs. Brussels would push an area of policy – the environment, say – and recommend a new agency to oversee compliance. Bosniaks would insist on one agency (at the central level) and Serbs would insist on two (at entity level, including one for Republika Srpska). Invariably, this was where the process got stuck.

So while the policy of conditionality was intended as the mechanism for stabilising the region, its effect was the opposite. In the absence of reform, the region remained stuck in political limbo, beyond the EU’s outer frontier. The EU began to lose control with the onset of the eurozone crisis, which brought the teleological project of building a European superstate to a halt. As firefighting and crisis management became the norm, the EU ceased to enlarge. With so many problems to solve, the last thing Europe needed was to admit a collection of corrupt, impoverished and ethnically divided states, all with potential veto powers and a treaty obligation to adopt the euro.

Indeed, many of the EU’s problems seemed to emanate from the Balkans. Most obviously, there was Greece’s mismanagement of its economy, which posed a mortal threat to the survival of the euro­zone and, by extension, the EU. But as Europe descended into recession, the issue of migration from Bulgaria and Romania also became a crucial political topic – and remains so today, as migrants and refugees from the chaos in the Middle East use the Balkans as a conduit to Europe.

In this context, no one was surprised when, in 2009, the German chancellor, Angela Merkel, publicly concluded that the EU needed to pause its policy of enlargement. All of this had an impact on the region, which interpreted it as Europe’s drawbridge closing. To all intents and purposes, the EU had reneged on its bargain with the western Balkans, which traded the prize of membership for good behaviour. All that remained was the prospect that one day, years in the future, after multiple reforms, the states of the region might join the EU, if it was in any position to enlarge and if it even still existed.

This changed the balance of risks and opportunities for those aspiring to join. Why continue with reform, especially when this implied serious economic pain? Was the EU even a desirable place to be? Greece’s example was hardly encouraging, nor that of Croatia, which squeezed its way into the EU in 2013 only to become the new sick man of Europe. Then the UK began to consider exit – hardly a vote of confidence.

Across the region, the reform process slowed even further. States such as Macedonia and Serbia shifted their focus towards the emerging economies of Turkey, Russia and China. Internal stability began to decline, aggravated by the recession that the EU exported to the region. Albania, Macedonia and others experienced mass demonstrations. Separatists began to renew their challenge to the American-imposed order, led by the Bosnian Serbs.

This is not to say there hasn’t been formal progress towards joining the EU. In the past couple of years, almost every country in the western Balkans has taken a step closer. Bosnia and Kosovo have been offered stabilisation and association agreements, the first step on the road to membership. Albania has been recognised as an official EU candidate. Serbia has opened membership negotiations. Montenegro, the most advanced country in the region, has closed several negotiating “chapters”. However, this bureaucratic progress does not necessarily reflect progress on the ground – in some cases, it signifies the opposite.

More precisely, the integration process has become a pretence that suits all sides. The EU can pretend the project of integration continues even as the eurozone and migration crises rage. And regional governments can pretend they are steering their countries towards a better future, for which they are richly rewarded by Brussels.

It is possible that a tiny country such as Montenegro will scrape into the EU on the back of this make-believe and Serbia will make some progress. But for other Balkan states, their journey towards Brussels will be more like that of Turkey, the eternal European aspirant. In reality, the western Balkans is once again losing its mooring.

***

The waning influence of the West has created an opening for new external powers, such as Russia, which has adopted a more active policy in the Balkans since the onset of the “new cold war”. Unquestionably, Russia is now a major influence on the region, especially in the Christian Orthodox countries of Serbia, Montenegro, Macedonia, Bulgaria and Greece. But its most significant involvement is in Bosnia.

In the past couple of years, Russia has feted Republika Srpska’s President Dodik, shielded Bosnian Serbs from accusations of genocide, called for an end to international supervision and, if media reports are correct, encouraged Bosnian Serbs to press their demands for independence.

Russia is not overtly trying to overturn the regional order. Instead, its aim is to bolster its alliances, deter the expansion of Nato and defend its economic interests in the Balkans. But regional disorder could still be the outcome. If Russia is cornered by the West over Ukraine, Moscow could trigger a serious regional crisis that embroils the EU and Nato, simply by giving a green light to the Bosnian Serbs.

A domino effect would then take hold. The departure of the Republika Srpska would open up the question of Serbia’s borders and encourage Kosovo’s Serbs to separate themselves completely from their country’s Albanian population. This would provoke Serbia’s Albanian minority, who live in an enclave adjacent to Kosovo, to make a similar break from Belgrade. Macedonia’s Albanians would then try to separate from their Slavic compatriots, fuelling the creation of a “Greater Albania”. Bosnian Croats would seek to integrate their territory with Croatia. And many in Montenegro would seek close relations with an expanded Serbian state. The West would undoubtedly refuse to recognise any of this to prevent the onset of violence but the facts on the ground would speak for themselves.

Any new Balkan conflict would draw in a wider cast of players. Russia would not sit by and let others determine the outcome of events; too much is at stake. The plight of Muslim Bosniaks and Albanians would draw in foreign jihadists, as happened in the wars of the 1990s – only in much greater numbers, given the upsurge in Islamism in Europe and the Middle East.

Meanwhile, several EU states would struggle to avoid entanglement. Croatia, which has recently adopted a more nationalist posture, would inevitably intervene in Bosnia on behalf of the Croat population. Bulgaria and Greece would take a keen interest in the fate of rump Macedonia after the departure of the Albanians.

All this leads to a sobering conclusion. As the EU loses its dominance in the Balkans, so the region’s unresolved nationalisms are returning to the surface on a bed of popular discontent. The Balkans have the potential to blow their problems back into Europe, entangling the EU in a new, potentially violent regional crisis. This may not happen tomorrow but, as the EU’s influence wanes, the day of reckoning draws ever closer.

Ideally, the EU would avert this possibility by fixing its internal problems, reviving the goal of enlargement and stabilising the region by means of integration, as has long been the plan. Yet, as matters stand, that looks like wishful thinking.

Timothy Less is the director of the Nova Europa political risk consultancy