In 1996 the CIA set up a special unit called Alec Station with the aim of targeting Osama bin Laden and the al-Qaeda network. It was headed by Michael Scheuer who found the Saudis less than cooperative. “When we set up the unit in 1996 we asked the Saudis for some basic material on bin Laden, like his birth certificate, his financial records – obvious stuff,” recalled Mr Scheuer many years later. “We got nothing.”

The CIA unit pursuing bin Laden kept on requesting this mundane but necessary information about their target from the Saudis for the next three years but got no reply. “Finally in 1999, we get a message from the [CIA] station chief in Riyadh, a Mr John Brennan,” Mr Scheuer said in an interview published in Kill Chain: Drones and the Rise of High-Tech Assassins by my brother Andrew Cockburn. “He said we should stop sending these requests as it was ‘upsetting the Saudis’.”

The story is important because John Brennan has been director of the CIA since he replaced David Petraeus in 2013 and last week he was once again avoiding any upset to the Saudis by telling the Saudi-owned al-Arabiya television station that the 28 pages in the 9/11 Congressional Inquiry relating to Saudi Arabia that have never been released contain “no evidence to indicate that the Saudi government as an institution -- or as senior Saudi officials individually -- had supported the 9/11 attacks."

This is not the impression of others who have read the report and dodges the question of indirect Saudi support for or tolerance of al-Qaeda in the past. But the important point is that in the 20 years between 1996 and 2016, the CIA and British security and foreign policy agencies have consistently given priority to maintaining their partnership with powerful Sunni states such as Saudi Arabia, the Gulf monarchies, Turkey and Pakistan over the elimination of terrorist organisations such as al-Qaeda, al-Nusra, Isis and the Taliban.

This contradiction between what is required to destroy the Salafi-jihadi Sunni fundamentalist networks and the need to maintain the alliance with these Sunni states goes a long way to explain the failure of the vastly expensive “War on Terror”. Commentators focus their criticism on Saudi Arabia and with good reason, but it was Pakistan that was crucial to the rise of the Taliban and Turkey to that of the extreme Islamists in Syria.

Mr Brennan does not take responsibility for this calamitous failure but he did admit its extent in revealing evidence to the Senate Intelligence Committee on Thursday. “Our efforts have not reduced the group’s terrorism capability,” he said. “The group would have to suffer even heavier losses of territory, manpower, and money for its terrorist capacity to decline significantly.”

Inside Isis secret tunnels Show all 7 1 /7 Inside Isis secret tunnels Inside Isis secret tunnels Network of underground tunnels was discovered by Kurdish forces after they regained the town of Sinjar in Iraq Inside Isis secret tunnels A member of the Peshmerga forces inspects a tunnel used by Isis militants in the town of Sinjar, Iraq Reuters Inside Isis secret tunnels An entrance to the tunnel used by Islamic State militants is seen in the town of Sinjar, Iraq Inside Isis secret tunnels The secret tunnels allowed militants to freely move underground Inside Isis secret tunnels The tunnels appear to be wired with electricity Inside Isis secret tunnels Some of the tunnels are 30 feet deep Inside Isis secret tunnels Concerns remain that parts of the tunnels are rigged with explosives

Isis is certainly weaker than it once was at its apogee as its divided but numerous enemies press forward on multiple fronts in Iraq and Syria. Its call for “lone wolf” attacks by Isis militants in their own countries is because it is more difficult for them to pass to and fro across the Turkish border, which has been progressively sealed off by the advance of the Syrian Kurds backed by US air power. But Isis leaders are all too aware that an atrocity like the one committed in Orlando - as in Brussels in March, and Paris last November - propels their movement to the top of the news agenda for weeks at a time and, for all the obloquy they receive, augments their strength and acts as a recruiting tool.

Isis is transmuting into more of a terrorist movement using guerrilla tactics as the onslaught against the self-declared Caliphate grinds slowly forward. The slaughter of civilians abroad is designed to mask a real weakening of the de facto Isis state compared to a year ago when it was still capable of launching offensives that captured Ramadi and Palmyra. President Obama is much more realistic than his critics allow in his claim that his campaign against Isis is showing real gains. Unfortunately, the credibility of his administration in convincing people of this is undermined by past exaggerations about victories over Isis that were swiftly contradicted by events.

What makes the current wave of terrorism different is that it is backed by the resources of a de facto state which, even in its current battered condition, can mobilise money, expertise, equipment and communications. The possession of territory with its own well-organised administration makes a great difference to the punching power of Isis when it comes to either terrorism or war.

Jo Cox MP speaks on Aleppo crisis

It is helped also by its enemies’ divisions, which are graphically on display at the moment in the battle for Aleppo where the CIA and the Pentagon pursue radically different policies. The Pentagon holds that its prime aim is to fight and defeat Isis, while the CIA maintains that Isis can be only be eliminated by first overthrowing President Bashar al-Assad and his regime. “The defeat of Assad is a necessary precondition to ultimately defeat [Isis],” a US intelligence official told Nancy Youssef of the Daily Beast. “As long as there is a failed leader in Damascus and a failed state in Syria, Isis will have a place to operate from.” This makes the dubious assumption that there would be less of a failed state in Damascus after Assad than there was before his fall.

Much the same argument is made by 51 US State Department diplomats in an internal memo calling for military strikes against Assad’s forces to compel him to abide by a ceasefire. It says that the Syrian government’s barrel bombing of civilians is the “root cause of the instability that continues to grip Syria and the broader region.” It calls for “a judicious use of stand-off and air weapons, which would undergird and drive a more focused and hard-nosed US diplomacy.” The mid-level diplomats, taking advantage of a State Department channel for officials to diverge from official policy without damaging their careers, say that at present Assad feels under no pressure to agree to a ceasefire, known as a “cessation of hostilities”, which excludes Isis and al-Nusra.

The diplomats’ demand for air strikes appears extraordinarily naïve and ill-considered since the last thing that Syria needs is yet more violence. Their protest avoids the problem that the Syrian armed opposition is dominated by Isis, al-Nusra and Ahrar al-Sham. It is understandable that the diplomats should denounce Assad for carrying out atrocities, but they should also accept that the only alternative to his rule is the Islamists. The US has found to its cost over the last few years that there is no moderate armed opposition fighting Assad which it can support aside from the Syrian Kurdish paramilitary army and some Arab units under Kurdish control.