Abu Bakr al-Baghdadi is the cartoonishly evil leader of the so-called Islamic State. His movement trades in fear – beheading aid workers, raping and enslaving women, using terror to scatter the conscript armies of Iraq and Syria. Yet he holds no fear for the Russian president, Vladimir Putin. For Putin, the rise of Isis has been an opportunity. Not many movements are hated by both Saudi Arabia and Iran, by China and the west – but Isis has that distinction. And by presenting himself as the leader to deal with it, Putin has transformed his global position.

Putin has been in a tight spot for the past few months. The Russian economy is struggling because of the low oil price and western sanctions. The war in Ukraine has undermined his international standing. By shifting his and the world’s attention from the Donbas to Syria, however, Putin is once again writing the script for inter­national politics, and forcing his opponents to recalibrate.

“The war in Syria was seen as a regional affair between Iran and Saudi Arabia-Turkey-Qatar, but now it is a bigger game between Russia and the west,” says Bassma Kodmani, a thoughtful expert on relations with the Middle East and former spokeswoman for the Syrian National Council, the opposition coalition-in-exile. By entrenching President Bashar al-Assad of Syria in power, Putin is also compelling western countries to engage with Moscow in a different way. Barack Obama, who had been trying to shun the Russian president, was forced to meet him at the UN General Assembly last month. Germany’s federal minister for economic affairs, Sigmar Gabriel, claimed that “you can’t stick to sanctions permanently on the one hand and ask for co-operation on the other hand”: so far, however, Chancellor Angela Merkel has refused to link the situation in Syria to sanctions on Ukraine.

Putin may be a capricious and unpredictable actor, but the prosecution of his Syrian military campaign shows that behind the tactical manoeuvring is a bigger strategic play: the desire to stop all sitting leaders – including himself – from being driven out of office by people power.

A red thread runs through many of his foreign policy decisions: an attempt to protect authoritarian governments from popular uprisings. Putin has long been troubled by the fate of Eduard Shevardnadze in Georgia, Askar Akayev in Kyrgyzstan, Viktor Yanukovych in Ukraine, Zine el-Abidine Ben Ali in Tunisia, Hosni Mubarak in Egypt and Muammar al-Gaddafi in Libya. In speech after speech, he has denounced the American ideal of promoting democracy.

In October last year at his flagship Valdai conference, where international experts discuss Russia and its role in the world, Putin said: “Instead of settling conflicts it [the advancement of democracy] leads to their escalation, instead of sovereign and stable states we see the growing spread of chaos, and instead of democracy there is support for a very dubious public ranging from open neo-fascists to Islamic radicals.”

As the Bulgarian political scientist Ivan Krastev has argued, “Western politicians imagine the Kremlin’s anxiety about colour revolutions is rhetorical, not real. But Mr Putin and his colleagues believe what they say: that street protests are stage-managed by Russia’s bitterest enemies.”

****

Putin believes he ended Russia’s chaotic democratic experiment under Boris Yeltsin and created the conditions for prosperity and stability under strong-arm rule. Now he aspires to play the same role on the world stage, becoming the ultimate counter-revolutionary leader. His advisers see this “war on coloured revolutions” – the term used to describe the uprisings in the former Soviet Union and the Middle East – as an organising principle for many of his decisions, the evil twin of America’s “global war on terror”.

Kadri Liik, a Russia expert who is my colleague at the European Council on Foreign Relations, frames the debate about Syria as a conflict between two competing world-views. “Obama believes in democratic stability while Putin has never experienced democracy as anything other than chaos,” she says. “His instinct is to put the genie back in the bottle and support dictators until they have re-established control.”

In Syria, Putin is executing his war on revolutions both on the practical level and as a battle of ideas. For some time, he has been building links with authoritarian powers in the Middle East – with President Abdel Fattah el-Sisi in Egypt and Prime Minister Haider al-Abadi in Iraq, as well as the Iranian government. And his Syrian gambit has bolstered the Assad regime at a time of increasing weakness. Putin’s campaign, supposedly against Isis, does not live up to its billing: most of the Russian attacks so far have been against other anti-Assad armed opposition groups (including forces backed by the west, Saudi, Turkey and Qatar).

There is a parallel between Putin’s plans for Syria and the long war he fought in Chechnya from 1999 to 2009. The first war in Chechnya, from 1994 to 1996, was between a moderate, largely secular opposition and the Russian state.

In order to win the second conflict, however, the Kremlin started to marginalise the moderates – starting with the legitimate president Aslan Maskhadov – while at the same time helping the factions that did not obey Maskhadov, and which committed kidnappings and were linked to the Middle East. Then, after the 9/11 attacks, Putin sold the Chechnya war to the west as “a common struggle with Islamic terrorism”. In Syria, a similar dynamic was already in motion – Islamist groups having gained the upper hand over the moderate rebels of the Free Syrian Army who helped launch the revolution in 2011 – but now Putin is accelerating it, using familiar tactics.

Russian planes have been targeting all of the anti-Assad groups to ensure that there is no strong, non-Isis opposition. At the same time, it appears as though Moscow has been actively helping Isis to swell its ranks. A report in the independent Russian newspaper Novaya Gazeta claimed that officers of Putin’s FSB (state security) have encouraged radicalised Muslims from Russia, and particularly the North Caucasus, to go to Syria, opening a “green channel” for travel that has made it possible for at least 2,400 fighters to make the journey (another 2,600 jihadis from central Asia are also believed to be in Syria). The newspaper claims that Russian agents are actively handing out special passports to jihadists to make it easier for them to travel.

This spring, Assad’s forces were being pushed back by the “Army of Conquest”, a military coalition of Islamist rebel groups, including Ahrar al-Sham and al-Nusra Front, that is supported by Turkey, Saudi Arabia and Qatar. In recent months, too, there has been a growing clamour to create safe zones that would be free from Assad’s bombs. By building up an ersatz air force for the Syrian leader, Moscow has made it harder for other countries to introduce no-fly zones or to use air power against Assad or Hezbollah, the Iranian-backed Shia militia from Lebanon. Not only does Putin’s deployment include fighter planes, attack helicopters and ground-to-air missiles, but western intelligence agencies claim he is building up an airbase near the port city of Latakia, as well as refurbishing Russia’s long-established naval base in Tartus.

Although Moscow has reportedly despatched a few thousand special forces troops to Syria, their main role will be to defend the military bases, conduct air strikes and train and equip the Syrian army, rather than engage in direct combat. They will also supply Damascus with the satellite imagery it needs to carry out its operations. Indeed, Putin does not seem to be investing enough to eliminate Isis. The Latakia base has space for only two squadrons, which would be able to fly roughly 500 sorties a month. Given the limited impact of a much greater number of western sorties this year, it seems unlikely Moscow will be able to defeat Isis or other rebel forces from the air.

Fyodor Lukyanov, the smart and well-connected chairman of Russia’s Council on Foreign and Defence Policy, believes that Putin’s goals are narrower than outright victory for Assad. In a recent article in the Russian press, Lukyanov wrote that Putin is trying to build in Syria the de facto equivalent of an “Alawite Israel” (the Alawites are the heterodox Shia sect to which Assad belongs). Lukyanov defines this as “a defensive secular enclave that – with outside support – would be capable of self-defence and serve as an obstacle to an uncontrolled spread of Isis”.

In this scenario, the Syrian army and Iranian-backed militias will get air support from Russia to secure the densely populated western part of the country, which is home to most of Syria’s industry and agricultural land, while much of the desert in the eastern part would be ceded to Isis. This new “core” Syria would be as dependent on Russia as Israel is on the United States and, similarly, would serve as a bridgehead for Moscow’s presence in the region. Although they accept Assad is unlikely to regain control of the whole of Syria, Putin and his advisers believe they can prevent his collapse, and that by doing so Moscow will put itself in a position to shape whatever comes next.

Some commentators have suggested that once Russia sees the dangers of getting bogged down in Syria, it might press for some kind of international settlement. By taking more ownership of the Syrian regime’s actions, Russia will be better positioned to influence Assad as well as less able to dodge the responsibility to try it.

The hope is that Assad would be persuaded eventually to end barrel bombing in the south and concentrate on “core” Syria, while agreeing to local ceasefires with other opposition groups. This would open the way for a managed decentralisation of the country into a Kurdish-controlled north, an Alawite west and a Sunni-dominated south. Local de-escalation could in time be followed by some kind of political process brokered by the main supporters of the various factions: Russia and Iran for the Assad camp, and Saudi Arabia, Turkey, Qatar and the west on the other side.

Yet, so far, the response to Putin’s intervention has been to escalate the crisis. Qatar has delivered a large shipment of arms through Turkey and Obama is under pressure from American hawks such as David Petraeus, the former commander of the Allied forces in Iraq, to increase US involvement. Even if they do not believe there is a military solution to the Syrian crisis, all participants would want to open talks from a position of strength, and so the net result is more violence.

More likely than a de-escalation inside Syria, at least in the short term, is a gradual lessening of conflict between Russia and the west, or at least a resumption of dialogue between the western and Russian military establishments. Soldiers on both sides have complained that levels of contact are much lower than they were at the height of the cold war. The buzzword in Washington is “de-confliction” – a concerted attempt to avoid Russian and western interventions leading to clashes.

****

While the gains in the Middle East are important to Putin, his most fundamental victory could be at a philosophical level. Putin smells American weakness and senses that Obama will eventually turn to Moscow rather than double down on his own failed campaign against Isis. In fact, in the week Russian planes started bombing, Obama’s former top adviser on the Middle East, Phil Gordon, published an essay calling for a rethink of the administration’s approach, which would open the way for an accommodation with Moscow.

The desperation in Washington is as apparent as Europe’s nervousness about the arrival of hundreds of thousands of refugees. If the west falls into the trap and goes to war alongside Putin and Assad, against not just Isis but all Islamist opposition groups, it will be the ultimate funeral pyre for the aspirations of the Arab spring, the talk about “being on the right side of history” and noble ideas such as the “responsibility to protect”. Western idealism will be exposed for all the world to see as the empty hypocrisy that Putin always thought it was.

Why is Putin so set on making this point? His biggest fear, I think, is not of colour revolutions in Damascus, nor even in Kyiv. It is of one taking place in Moscow. Putin is still haunted by the winter protests of 2012 that were provoked by his return to the Kremlin as president for a third term.

Much of his foreign policy since has been driven by this experience. In February 2014, when Yanukovych was hounded into exile by protesters in Ukraine, Putin feared he could be vulnerable. If his Syrian gamble does pay off, it might just force the west to recognise the benefits of autocratic stability.

But the domestic politics are not risk-free. There is a debate in Moscow about the danger of getting bogged down in Syria as the US did in Iraq. More resonant than Iraq in the Russian imagination is another quixotic campaign in the desert: the disastrous invasion of Afghanistan in 1980. Leonid Isaev, writing last week in the Moscow-based daily Vedomosti, said the Afghan campaign brought “ten years of dismay to Russia, and nothing else apart from coffins”.

A recent poll found that only 14 per cent of Russians believed their country should provide direct military support for the Syrian government by sending in troops. That is why the Kremlin is briefing that most of the sorties are being flown by Syrian pilots.

The consensus is that Putin is a brilliant tactician but a terrible strategist. Yet since the beginning of the Ukraine crisis, he has been using his unpredictability to increase his leverage over the west, keeping everyone from his closest aides to foreign governments on their toes. On a visit to Moscow in April 2014 after the annexation of Crimea, I was told by advisers close to the Kremlin that Putin’s favoured concept was “manageable chaos”. Many commentators in the west are predicting that the Russian campaign in Syria will go wrong – and soon become unmanageable. But Putin has been underestimated before. His fight against Isis may not simply score a point against western values. It may help to save his regime.

Mark Leonard is the co-founder and director of the European Council on Foreign Relations