It’s worth saying aloud what everyone already privately accepts: this is the new normal. The attempted bombing of a London Underground train at Parsons Green in west London on 15 September was the fourth terrorist outrage linked to Islamic State (IS) since the March attack on Westminster Bridge and parliament.

Over the same period, another six plots have been foiled and 105 people have been charged with terrorism offences. The pattern is similar in France, Belgium and Germany, where attacks and preventative raids have become part of daily life.

IS quickly claimed the failed bombing at Parsons Green, saying it was carried out by one of the group’s “detachments” or “units”. It is easy and tempting – and indeed common – to dismiss such claims as naked opportunism; a desperate attempt by a struggling terrorist organisation to assert its relevance.

Perhaps there is an underlying psychological factor behind this – that we simply cannot accept the impact IS is having on the security of our country and continent (and, more broadly, the world). Yet, this view is also belied by the facts.

In almost every case where IS claims an attack, a link of some sort between the perpetrator and the terror group can be found. The arrest in June of a 23-year-old Syrian man in Germany who was linked to Amaq – one of IS’s official media arms – is instructive.

When German authorities swooped on the man, identified by prosecutors only as Mohammed G, they gained invaluable insights into the way IS claims responsibility for an attack. IS had taken credit for several attacks through Mohammed G, but were sceptical when he told them a man in Sweden had burned down a Shia mosque in the group’s name. Although IS conducts a vicious sectarian campaign in Iraq and Syria against Shia Muslims for their supposedly heretical beliefs, the terror group initially refused to accept the arson.

The Swedish man was told he would have to prove himself by making a video offering a bay’ah to Abu Bakr al-Baghdadi, the leader of IS and self-styled caliph. This is extraordinary for two reasons. The first is that it demonstrates IS does not claim every attack, even when “offered” one. The second is that it reveals just why IS has been much more successful than al-Qaeda was at mobilising terrorists abroad.

The bay’ah is an oath of allegiance given by Muslims to a rightful authority or leader. From the moment Baghdadi first appeared on the steps of the al-Nuri mosque in Mosul, Iraq, to declare the establishment of the caliphate in 2014, he electrified radical Muslims across the world. As leader of the caliphate, Baghdadi told his constituency of followers that he was the caliph, meaning “successor”. But successor to whom and in what capacity?

His followers understood this to mean he was a successor to the Prophet Muhammad in guardianship over the Muslim world. Baghdadi was essentially arrogating for himself the honorific title of amir al-mu’minin, or commander of the faithful. It was all his radical constituency needed to hear. Finally, they had someone to whom they could offer their duty.

It is the potency of these ideas that has made IS so menacing, driving its ability to inspire attacks across the world. The group’s followers feel a sense of loyalty and duty to their beleaguered caliph. His will must be obeyed.

Beyond the ideological aspect is the practical one. Although al-Qaeda frequently called for attacks across the West, it simply couldn’t appeal to young people in the way IS does. Despite its crimes and excesses, al-Qaeda strived to present itself as a principally religious organisation. That meant peppering its books and videos with verbose, arcane references to dense pieces of Islamic theology.

By contrast, IS thinks in 140 characters, not 140 pages. Its propaganda is based around a series of emotive propositions, forcing supporters to make easy and binary choices. It relies on the promise of panacea, rather than a sophisticated understanding of theology. Its propaganda resonates particularly well with young people who are more likely to accept simple prescriptions for a world that is less stable today than at any point since the end of the Second World War.

The relationship between IS and its supporters is complex, however, as evidenced by the case of Omar Hussain, a British fighter from High Wycombe who is among the group’s most zealous members.

Hussain has been a consistent critic of IS, accusing its leadership of ignorance and naivety. Yet he remains a loyal soldier of the caliphate and is serving on its front lines in Raqqa, where the the Western-backed Syrian Democratic Forces are making significant gains in their attempts to liberate the Syrian city.

To Hussain, it is the institution of the caliphate that matters most. This is a religious obligation and represents the suzerainty of God over his people. By contrast, IS is just a group and Baghdadi is just a man. Everyone makes mistakes and has shortcomings and limitations, but this neither invalidates the caliphate nor the caliph’s right to obedience.

Herein lies the key to understanding the intractability of the terrorist threat posed by IS. Even if the terror group crumbles or makes decisions that annoy its base, it can still mobilise a constituency who remain hostages of history.

It yearns for a new l’ancien régime, a restoration of the Islamic Bourbons, and feels it has seen a glimmer of it with Baghdadi. This explains the framework of ideas that have given rise to the new era of terror in Europe, polarising communities and creating fear and suspicion.

As arrests are made in the aftermath of the Parsons Green attack and we resign ourselves to grappling with a menace that looks set to last for at least a generation, mitigating the deadly outcomes of terrorist attacks is only one of the challenges we face.