On 16 June​ , Kurdish militiamen, with the support of US airstrikes, captured the town of Tal Abyad in northern Syria, a major crossing point on the Syrian-Turkish border. Its fall is damaging to Islamic State: it cuts the road linking the caliphate’s unofficial Syrian capital at Raqqa, sixty miles to the south, to Turkey and the outside world. Down this road have come thousands of foreign volunteers, many of whom became suicide bombers. Now the movement is all the other way: some 23,000 Arab and Turkmen refugees have fled into Turkey to escape the advancing Kurds. Some passed children over tangles of barbed wire before following through a hole cut in the border fence. Recep Tayyip Erdogan, the Turkish president, accused Western powers of using airstrikes to support Syrian Kurdish ‘terrorists’. Towards the end Islamic State seems only to have had some 150 fighters in Tal Abyad. It didn’t send reinforcements because it knew the fall of the town, surrounded by Kurds on three sides, was inevitable.

This is the latest Kurdish victory in the ‘war within a war’ being waged in the north-east corner of Syria between Islamic State fighters and the military wing of the PYD, the Syrian Kurdish party that rules the Kurdish enclaves along the border. I was 15 miles east of Tal Abyad at the end of May when Kurdish women soldiers in the frontline were speaking guardedly of the coming offensive. There was a constant roar of aircraft overhead, probably American, but I didn’t hear any airstrikes. Nujaan, a 27-year-old veteran of the Kurdish women’s militia, said they were advancing steadily west towards Tal Abyad; there had been fighting that morning, and several Kurdish soldiers had been killed or wounded. In the de facto Kurdish capital, al-Qamishli, I spoke to Sehanok Dibo, an adviser to Saleh Muslim and Asya Abdullah, the leaders of the PYD, who confirmed that Tal Abyad was the next target: ‘We hope to liberate it soon.’ He repeatedly insisted that it wasn’t only Kurds who were fighting Islamic State, but also members of the Syrian armed opposition to Bashar al-Assad. I wondered if this was a piece of propaganda, meant to make the PYD sound less ethnocentric and more acceptable to the Americans. I pressed Dibo on just how many of these pro-Kurdish rebels were to be found fighting in the frontline, since I hadn’t seen any. Would they be enough to fill the not very large room we were sitting in, I asked. ‘Maybe two rooms,’ Dibo finally admitted.

Despite the PYD’s denials, and probably their best intentions, the conflict in north-east Syria has many aspects of an ethnic war: the Kurds are driving out Sunni Arabs, whom they accuse of being Islamic State supporters. Those Arabs who flee are seen as demonstrably in league with the enemy: those who stay are suspected of belonging to ‘sleeper cells’, waiting their moment to strike. The Kurds say that they and their ancestors have lived in the area around Tal Abyad for twenty thousand years; the Arabs, they maintain, are recently arrived settlers, beneficiaries of a Baath Party campaign in the 1970s to establish a nine-mile-wide Arab Belt along the border. Arabs who are now being evicted from their homes say the Kurds are telling them to ‘go back to the desert’.

For the 2.2 million Syrian Kurds, a tenth of the Syrian population, the capture of Tal Abyad has enabled them to connect two of their three enclaves, which they call Rojava, or West Kurdistan. The largest enclave, or canton as the Kurds call it, is known as Jazira or ‘the Island’, because of its position between the Tigris and the Euphrates; it is an isolated statelet flanked by Iraq to the east and Turkey to the north. Its main city is Qamishli, which feels a long way from the war. This is a fertile and largely self-sufficient region of wheat fields and oil wells, though few are still operating. Further west is the canton that surrounds the devastated town of Kobani, which Islamic State failed to capture despite a four-and-a-half-month siege that ended in January when its forces finally withdrew after losing an estimated one thousand fighters thanks to some seven hundred US airstrikes and fierce resistance by the Kurdish People’s Protection Units (YPG). With Tal Abyad in their hands, the Kurds now control a 250-mile-long swathe of territory along Turkey’s southern frontier, an achievement that is likely to cause dismay in Ankara.

The fall of Tal Abyad is significant, but it is still only one more episode in the war now engulfing Syria and Iraq, a war in which military success seldom brings final victory much closer. It is, some argue, another Thirty Years’ War. The problem in Iraq and Syria today, as in Central Europe in the 17th century, is that there are too many players, inside and outside the countries where the fighting is taking place, who can’t afford to lose and will do anything to win. In Qamishli, Sehanok Dibo told me that ‘the balance of power in Syria can change abruptly if any of the foreign countries involved here changes its stance.’ Last year that change happened when the US started to support the YPG in Kobani with airstrikes. But the situation could be transformed again, with disastrous consequences for the Kurds, if the Turkish army, as it may, crosses the border to set up a buffer zone on Kurdish-held territory.

Despite the PYD’s insistence that it is more than just a Kurdish nationalist party, sectarian and ethnic loyalties are at the heart of the multi-layered civil wars convulsing Syria and Iraq, whatever the original cause of the conflict. In both countries, the collapse of central government has exposed and sharpened differences between Arab and Kurd, Sunni and Shia, Muslim and Christian, secular and religious. And as Syrians and Iraqis live in a permanent state of war, these differences are now almost always settled violently. From the Iranian border to the Mediterranean, civilian populations are fleeing their towns and villages whenever the army or militia defending them is defeated. Islamic State is more violent than other movements, publicising as it does the ritual slaughter of Shia, Yazidis and anybody else who opposes it. But Jabhat al-Nusra, the al-Qaida affiliate that is backed by Saudi Arabia, Turkey and Qatar, isn’t far behind, forcibly converting Druze villagers to its extreme version of Islam; on 10 June it shot dead twenty Druze in one village, Qalb Lawzeh in Idlib Province. Meanwhile, the Syrian government uses barrel bombs and every other kind of ordnance to pound into rubble, regardless of civilian casualties, any built-up area resisting it. Many of the outer suburbs of Damascus, once held by the rebels, are today in ruins: they look like pictures of Hamburg and Dresden in 1945.

Communal suspicion and hatred have gone too far to be reversed. In May, I reported from Mount Abdul Aziz, a partly forested area south-west of Hasakah City, which the YPG had just captured after several days’ fighting. I asked the YPG commander, General Garzan Gerer, about the problems he had faced in seizing the mountain. He said there had been two difficulties: one was the mountainous terrain and the other was that ‘many of the local villages are Arab and they often support Daesh’ – the Arabic acronym for Islamic State. He didn’t think many of the villagers would be coming back. This turned out not to be entirely true: as we drove away from the front, we saw a family of Arabs carrying their belongings back to their house in an otherwise deserted village. They waved with exaggerated enthusiasm at our vehicle, as if uncertain about how they would be treated by the victorious Kurds. Many Iraqi and Syrian men in their twenties have done nothing all their lives but fight. One such man is Faraj (not his real name), a 29-year-old Islamic State fighter who comes from a Sunni Arab village between the cities of Hasakah and Qamishli. He was one of the militants in Tal Abyad waiting for the final assault as the YPG forces closed in. A Kurdish colleague from the area contacted Faraj via WhatsApp and transcribed the conversation for me. His replies to questions were sometimes confused and disjointed, but when he spoke of the town’s impending loss he was calm, possibly because, though a graduate of the Faculty of Education at Hasakah University, fighting is all he has known over the last four years. ‘So what if we lose the Turkish border,’ he said. ‘I think Islamic State still has open borders with Iraq. It will remain strong and, according to our commanders’ reports, it may lose some battles, but it has its own strategies for winning the war.’ He is philosophical about American airstrikes, saying that they can’t achieve much without ground forces: ‘I think that Islamic State is winning not losing.’

He may well be right. On 17 May, Islamic State captured Ramadi, the capital of Anbar province, seventy miles west of Baghdad, and a few days later took Palmyra, at the centre of transport routes east of Damascus. This ended a period when there had been a wave of wishful thinking in Western capitals and Baghdad that the self-declared caliphate was weakening, unable to advance because of US airstrikes and squeezed economically by the hostile states encircling it.

But there is little evidence that the caliphate has got any weaker in the year since Abu Bakr al-Baghdadi declared its birth in Mosul on 29 June 2014 after capturing the city. The fall of Tikrit on 1 April this year, after a month-long attack by the Iraqi army and Shia militias backed by US airstrikes, was presented as a sign that Islamic State might be unable to withstand the military pressure it was under on many fronts. The Pentagon spoke of retaking Mosul. The Iraqi prime minister, Haider al-Abadi, announced triumphantly that ‘the next battle’ would be to recapture the giant Anbar Province. In fact, the opposite happened: the Iraqi army offensive had scarcely begun when it was overrun by an Islamic State counterattack that seized Ramadi as elite units of the Iraqi security forces fled.

From the point of view of the Baghdad government and its backers in the US and Europe the overall military picture is grim. Part of the problem is the failure to rebuild the Iraqi army, which disintegrated so humiliatingly last year when it lost much of northern and western Iraq. According to a senior security official, an army that once had 360,000 soldiers on its books – though many were ‘zombie’ or ‘virtual’ soldiers who didn’t exist, with their salaries drawn by officers and defence ministry officials – now has between 10,000 and 12,000 combat-ready troops. These units, among them the so-called Golden Division and Swat forces from the Interior Ministry, have been rushed like a fire brigade from crisis to crisis until their soldiers are exhausted and demoralised by heavy losses. For actual military might, Baghdad now has to rely on Shia militiamen who are willing to fight, but are in part controlled by Iran.

After exposing themselves so disastrously to US air power at Kobani, Islamic State commanders have changed their tactics. They are now giving up territory if it can’t easily be held, before launching surprise counterattacks elsewhere. This new approach means not fighting to the last bullet unless conditions are favourable. They make many pinprick assaults up and down the weakly held frontlines; the Iraqi Kurdish front with Islamic State is six hundred miles long (the whole Western Front in 1914 was four hundred miles). These attacks often fail, but they are partly diversionary, intended to keep the enemy guessing about when and where the main assault will come. Foreign volunteers are often used as cannon fodder. Islamic State, for all its exaltation of martyrdom, is more careful now with the lives of its Iraqi and Syrian fighters. The number of local fighters has been growing rapidly because Islamic State has been conscripting all young men over the age of 16 in the area it controls – an area about the size of Great Britain with a population of six million. The recruitment drive has enabled it this year, more easily than last, to fight on multiple fronts, from the outskirts of Baghdad to the suburbs of Damascus.

Faraj didn’t say whether he expected to survive the fight for Tal Abyad. On other occasions, experienced Islamic State veterans have slipped away at the last minute. But Faraj’s account of why he joined Islamic State and is loyal to its cause must be true for others: a great many reasonable Syrians and Iraqis have joined this fanatical movement, despite its barbaric and very public cruelty, outlandish ideology and cult of death, and stay with it despite the likelihood of temporary defeat. ‘Even if this happens,’ Faraj said, ‘I still believe that we are right because most of us are not fighting for women or money; we are fighting because both the regime and the opposition failed us, so we need an armed organisation to fight for our rights.’

Until last year, Jabhat al-Nusra was strong in Kurdish areas, but was squeezed out in heavy fighting by the YPG on one side and Islamic State on the other. Faraj and his extended family joined al-Nusra in the year after the Syrian uprising began in 2011. ‘At first we dreamed of having a revolution and gaining our liberty,’ he said, ‘but unfortunately the popular movement was not well organised and was manipulated by neighbouring countries such as the Gulf states, so revolution turned into jihad.’ He says that to fight back against the regime the rebels had no choice but to turn to a religious movement that appealed to the conservative people of eastern Syria. Another motive was revenge: for ‘the oppression and injustice of the regime over the last forty years that weighed down our souls’.

In July 2012 the Syrian army almost entirely withdrew from the three Kurdish cantons to reinforce regime strongholds elsewhere. It held on to a couple of small symbolic enclaves in Qamishli and Hasakah so that the regime in Damascus could claim still to have a presence everywhere in the country, even though it was no longer in control. ‘When the Kurdish forces took over we felt we had gained nothing by our revolution,’ Faraj said. ‘They were just as oppressive as the regime.’ He fought back as a member of al-Nusra until it was defeated by the YPG. Islamic State then came to his home village, where he says that ‘members of al-Nusra were given a choice of joining Islamic State or leaving the village.’ He was one of five who decided to join, two locals and three Tunisians. This February, Kurdish forces entered the village and he was sent on a mission to Raqqa while the others stayed to fight: ‘They resisted for five hours, but were only four men against thirty so the three Tunisians were killed and only the local fighter escaped.’ Faraj returned to the area from Raqqa and spent a month contacting villagers he knew.

At this point, Faraj says he met many foreign fighters from Britain, Turkey and France, some of whom had learned Arabic well. He wasn’t impressed by them: ‘I know many fighters from the Gulf states, Europe and Australia who are fighting for arms, fame, women and money.’ When he asked volunteers from Europe why they were in Syria some told him that their lives were miserable at home or that they had simply been bored. Many had found ‘spiritual happiness in Islam’, but Faraj said that they were often recent converts who didn’t seem to know much about Islam or local customs. The foreign fighters, he said, were mostly used for suicide attacks and propaganda, ‘while the locals are used for fighting’.

This is the pattern across the territory controlled by Islamic State. It’s often difficult to know how many foreign fighters are present in a battle: Kurdish and Iraqi army commanders like to claim that almost all the fighters facing them are heavily armed foreigners from the Muslim world or Western Europe. This was the official line when the Iraqi Kurdish peshmerga was defeated by Islamic State last August. But when I talked to Christian and Yazidi villagers who had seen their attackers before they fled, they said the fighters were all Iraqis, few in number and driving unarmoured vehicles. There are, however, some parts of the front that foreigners do hold. At Mount Abdul Aziz, Kurdish fighters showed me a notebook that had been found in an Islamic State headquarters: in neat handwriting it listed the Arabic equivalent for various common Russian words. On one page there was the plan of a room with arrows pointing to a table, chairs and other items with their Arabic names noted. Presumably, the owner of the notebook had come from a Russian-speaking Muslim country in the Caucasus or Central Asia.

What makes Faraj’s account of his life and views so interesting is that he isn’t a defector or a propagandist. He is somebody with a deep hatred of the Assad regime who joined the organisation that was most able to fight against it. He told the story of his former leader or emir, an Iraqi Kurd with the nom de guerre Abu Abbas al-Kurdistani, who had recently been killed in battle. Faraj asked him why he had joined Islamic State and Abu Abbas replied that he had been imprisoned by the Kurdistan Regional Government for four years without a fair trial. ‘Corruption and torture,’ Faraj said, ‘had pushed him to find any organisation that gives him the opportunity for taking revenge. Our emir’s pain was similar to ours. We all fight as a reaction to the tyranny and injustice we had known before. Islamic State is the best option for oppressed people in the Middle East.’

The capture of Tal Abyad by the Kurds may well lead to a fresh wave of speculation that Islamic State is going into decline. But, like most of the other participants in the civil wars in Iraq and Syria, the self-declared caliphate is too well rooted to disappear. Its quasi-guerrilla style of warfare makes the loss or gain of a single town or city less significant than it may appear. Its slogan, ‘the Islamic State remains, the Islamic State expands,’ is still true.