The latest video from Isis introduces a new British executioner, a successor to ‘Jihadi John’, and it is a classic of the genre: bombastic, pompous, ridiculous yet terrifying. ‘O slave of the White House, O mule of the Jews,’ says a man in a ski mask, addressing David Cameron, ‘how strange it is that the leader of a small island threatens us with a handful of planes. Only an imbecile would dare to wage war against a land where the law of Allah reigns supreme.’ He has a cold, arrogant look in his eyes and brandishes a pistol held sideways, aping American ‘gangsta style’. Kneeling before him are five men in orange jumpsuits identified as ‘British spies’. In fact they may just be Syrian journalists whose ‘crime’ was simply trying to do honest reporting about the so-called Islamic State. I have a sickening feeling that one of them supplied pictures I used in a piece for the BBC when I was on the Syrian border last year. Of course, the men are killed in the video.

We have become wearily used to such brutal ‘propaganda by deed’. Isis’s new year message to Britain is much the same as it was last year: Be afraid! We are coming for you! It is still chilling. But Isis is not the force it was 12 months ago, and its video nastiness seems now more a sign of weakness than strength. The group has suffered a string of defeats in recent months. The tally so far includes the town of Kobani in northern Syria; Sinjar, in Iraq, which fell to the Kurds in November; and Ramadi, taken back by the Iraqi army two weeks ago. Fighters who promised to love death as we much as we love life appear to be losing their nerve. In Sinjar, there are reports that two ‘brigades’ of fighters deserted. In Ramadi, there was no bitter struggle to the last suicide vest. Instead, Isis melted away, leaving behind booby traps and car bombs to slow the Iraqi advance. So far, US officials claim, American bombs have killed 20,000 Isis jihadis.

A friend of mine, a diplomat, has developed sources within the group and he describes a very different Isis from the one we see in their propaganda videos: ‘Morale is plummeting within Isis, especially among foreign fighters,’ he says. ‘Many European foreign fighters in particular are packing it in. Many want to defect. Whole units have just gone away in Iraq… the Islamic State is in crisis.’ There was a struggle within the Isis leadership between hawks and doves, the diplomat said, with the hardliners gaining the upper hand. But they were frustrated at being unable to mount big, shock attacks as they did in 2014 — because of western bombing. ‘Inherent Resolve [the US-led campaign against Isis] is much more effective than it is given credit for… the further expansion of Isis has been stopped.’

The leadership is still striving to attract new recruits to ‘God’s Kingdom on Earth’. A video in English and French shows an Isis loyalist taking his three small daughters to the shops — all well-stocked — and then to a fairground. ‘Brothers and sisters, come to caliphate,’ says the narrator, making a special appeal for engineers, doctors and nurses.

Some do find they like life under the caliphate. A British recruit to Isis described the attraction to me once. ‘It’s a really good feeling. You can go around without anybody trying to harass you [for being a Muslim],’ he said. ‘I don’t miss a thing about Britain. Here, I can drive, I don’t need a licence. If I want to watch TV, I don’t need a licence for that either. I can walk around with a Kalashnikov, with an RPG [rocket-propelled grenade] if I want to. It’s total freedom, thanks be to God. It’s like I’m flying.’

But as my diplomat friend says, many others have become disillusioned. Last year I met an activist called Abu Ibrahim who runs a network that gets people out of Isis territory. ‘Most of those who go to the caliphate are true Muslims,’ he told me. ‘They are shocked when they get there and see what things are really like. It is horrible — for them it is either flee or commit suicide.’ He estimated that 40 per cent of foreign fighters wanted to leave. But desertion involves a terrifying risk. In one six-month period last year, 400 fighters were executed for disloyalty, according to his own source inside Isis. He managed to take out a handful before they were killed, including three Britons: two men and a woman.

Abu Ibrahim showed me a photocopy of a French passport belonging to a woman in a hijab. He had smuggled her out, too. She had gone to Syria to join the jihad but learned — along with other women — that her duty would simply be to bear the next generation of jihadis. She was promptly married off. When her new husband — another French citizen — was executed for refusing to obey orders, she was put into a prison for women, all foreigners, all widows of executed men or wives of men held in Isis dungeons. There were some 300 women in the prison, she remembered. They were watched all the time. Two people were executed for trying to smuggle in mobile phones for them. The only way out permitted was by marriage, though she was eventually rescued by Abu Ibrahim’s network.

Her husband had been killed for refusing to join the bitter side-war against the Islamic State’s rival, the Nusra Front, al-Qaeda’s loyalists in Syria. Many others have been killed for the same reason. They had joined Isis to kill the unbelievers of the Syrian regime, not other Muslims classed by the leadership as sinners. This is a real weakness for Isis: they are locked in a continual struggle for members with Nusra and other Salafi-jihadist groups in Syria. Fighters defect to Isis and then defect back again. Often, whole units and their commanders change sides in this way. Isis is not a monolith but a coalition. That coalition includes criminal gangs and groups of tribal fighters whose sheikhs have pledged loyalty to the caliphate. I watched the Kurds battle Isis for possession of one town in eastern Iraq. Kurdish intelligence said they were facing a force of about 800. That consisted of around 650 locals and some 150 outsiders, Isis shock troops led by a commander who brought with him a bag of cash for the sheikh and promises to protect the tribe against all enemies.

When Isis looked unstoppable 18 months ago, the criminal gangs and the Sunni tribal leaders all wanted to sign up with a winner. Now some at least are looking for the next ‘strong horse’. If Isis is driven away from the Syrian town of Aleppo — their next predicted defeat — it will be because of this phenomenon of groups changing sides. The collapse there could be quite rapid. After that, if Ramadi really has taught the Iraqi army how to do street fighting, the next town to fall could be Fallujah. That might take a bit more time, but to the optimists it all seems as if Isis is being marched towards inexorable defeat.

It is too soon, though, to say that the caliphate is done for. Can it really be correct, for instance, that 20,000 Isis fighters have been killed by American bombs? The CIA always estimated that Isis could put 30,000 fighters in the field. On US figures, then, Isis has suffered a rate of attrition of two thirds. And since the CIA estimate has not varied, they have supposedly been able to replace two thirds of their number with new recruits. That is unlikely. Isis probably had more fighters than the CIA thought; the casualties are probably less than US officials are claiming and, crucially, they continue to attract new recruits, alienated young men and women who want to fight for a cause and live the life set out by God to Mohammed.

The bombing, too, is sometimes counterproductive. Isis are not an organised army based in large barracks easy to spot from 10,000 feet up. They are a ragtag militia living and moving among the population. However careful the targeting, there have been civilian casualties — and Isis makes the most of those for propaganda purposes. In the worst recent incident, last month, activists say that some 30 civilians — half of them women and children — died in an airstrike on the village of Al Khan in Syria. A year ago, a missile hit the Isis prison in the town of Al Bab. It probably looked like a barracks from the air, with armed men coming and going. But civil defence volunteers in Al Bab say they pulled 90 bodies from the rubble, most of them — since this was a prison — people who had been enemies of Isis. The Sunni Arabs who have to be won over in the fight against Isis are understandably angry about the bombing.

Abu Ibrahim — the activist smuggling people out of Isis territory — told me the war would never be won from the air, nor with the US-led coalition’s existing allies on the ground. ‘How many airstrikes have they carried out? 1,000? 2,000? 3,000? (The Pentagon says it is actually 9,400.) If a quarter of that effort was used for the ground battle, Isis would have already been crushed.’ The problem, he said, was that most of the victories against Isis have been by the Kurdish militias taking back Kurdish territory, where there was no Sunni Arab population that Isis could enlist to mount a defence. In Syria, outside the Kurdish areas, there are really no significant armed groups the Americans can count on as allies. That is what led the former US Army general and CIA director David Petraeus to make the remarkable suggestion of an alliance with ‘moderate’ members of the Nusra Front: that is, an American alliance with al-Qaeda.

In Iraq, it is true, the government has taken back Ramadi, and before that Baji and Tikrit. But the Iraqi army is as much a business as a military force. Many of its officers are there to enrich themselves, happy to leave the fighting to Shia militias (paid for and directed by Iran). It is far from certain whether the Iraqi army would be capable of taking back Mosul, a city of a million and a half, well defended with tonnes of weaponry captured by Isis when government forces fled in 2014. The real problem in Mosul, and elsewhere, is that many Sunni Arabs quite like being ruled by Isis. Or at least they prefer Isis to what they fear would be murder and pillage by Shia death squads sent by a government in Baghdad they still think of as sectarian.

Victory over Isis depends, as it always did, on winning over ordinary Sunnis. It is a political problem as much as a military one. But if Sunnis ruled by Isis can be persuaded to abandon the caliphate, taking with them the ‘fair-weather jihadis’, the hardest part of the battle would still be to come. The die-hard loyalists would be left, including many British jihadis. One British official who deals with this issue compared the core of Isis to the SS in Nazi Germany: fanatics who would never give up. These are people who believe literally that they are about to fight the final battle, ordained by prophecy and scripture, marking the End of Days. (It will be fought against the West and its allies and will take place in Syria.)

The death throes of the caliphate will therefore take time. During this period, as Isis has warned, the West can expect more attacks. At the end of the ‘new Jihadi John’ video released this week, a cherubic little boy, speaking with a British accent, promises to bring the jihad back to the UK: ‘We will kill the kuffar [infidels] over there.’ Hundreds of British Muslims have joined Isis. If or when the caliphate is smashed to pieces, some of them will be coming home.

Morale is plummeting, especially among foreign fighters, and many are packing it in. Many want to defect

Paul Wood is a BBC Middle East correspondent and has reported from Libya, Syria and Iraq.