Millions of black-clad Shia pilgrims are converging on the holy city of Kerbala for the Arbaeen religious commemoration, the largest annual gathering of people anywhere on earth. Walking in long columns stretching back unbroken for as much as 50 miles, sleeping and eating in tents erected by supporters beside the road, the event has become an overwhelmingly powerful display of Shia belief and solidarity.

The Arbaeen coincides this year with the final defeat of Isis, the movement that slaughtered Shia in their tens of thousands and aimed to overthrow the Shia-dominated government in Baghdad. The Syrian army announced today that it has captured the last Isis-held town in Syria, Albu Kamal, its victory coming a few days after Isis was driven from western Iraq.

Arbaeen is the living symbol of the rise of the Iraqi Shia, a highly significant development in the Middle East, but it has happened only recently. Karim, 48, a tribal leader from Najaf, who provides free food for the pilgrims, recalls that when he first took part in an illegal Arbaeen walk under Saddam Hussein, “we had to take a roundabout route by the river [Euphrates] and try to keep hidden because, if we were caught, we would put in prison or executed”.

The Arbaeen has provided many modern-day Shia martyrs, murdered by Saddam Hussein, al-Qaeda and Isis, but its purpose is to mourn the martyrdom of Imam Hussein, the revered Shia leader, killed in the battle for Kerbala in AD680. The long ritual walk to his golden-domed shrine in that city – some walkers spend 10 or 12 days on the road from Basra or Kirkuk, others two or three days from Najaf – comes on the 40th day of the mourning period as religious fervour reaches its peak among the faithful.

Shia cities, towns and villages all over Iraq empty out during a 20-day period as their people take to the roads in an elaborately organised and well protected mass movement not seen anywhere else in the world. Estimates vary of the total attending, from highs of 15-17 million to a low of 6-7 million, but it includes at least two million Iranians whose numbers are easier to calculate because they require documentation to enter Iraq. Mohammed al-Hilli, the author of a book entitled The Arbaeen: the Walk, says that “the city of Kerbala can only contain two or three million people at one time, but, since pilgrims are coming and going over a long time, the total attending will be much higher.”

Shia Muslim pilgrims gather for the Arbaeen religious festival Show all 20 1 /20 Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims gather for the Arbaeen religious festival Shia pilgrims praying at the Immam Hussein shrine AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia pilgrims gathering in front of the Immam Hussein shrine AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia pilgrims mourning in front of the Immam Hussein shrine AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia pilgrims praying at the Immam Hussein shrine AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia pilgrims pray at the Imam al-Abbas shrine during the commemoration of Arbaeen in Kerbala REUTERS Shia Muslim pilgrims gather for the Arbaeen religious festival Shia pilgrim mourning AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia pilgrims pray at the Imam al-Abbas shrine REUTERS Shia Muslim pilgrims gather for the Arbaeen religious festival Shiite pilgrims gathering in front of the Immam Hussein shrine AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia faithful pilgrims gather between the holy shrine of Imam Hussein and the holy shrine of Imam Abbas AP Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims gather in the southern Iraqi city of Karbala AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival A man flashes the V-sign as Muslim pilgrims gather at the Immam Hussein shrine AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims gather the Immam Abbas ibn Ali shrine in the southern Iraqi city of Karbala AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims gather during the commemoration of Arbaeen in Kerbala REUTERS Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims ahead of the Arbaeen religious festival, which marks the 40th day after Ashura AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival The holiday marks the end of the forty day mourning period after the anniversary of the martyrdom of Imam Hussein AP Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims gather ahead of the Arbaeen religious festival AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia faithful pilgrims march to Karbala AP Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims gather at the Immam Abbas ibn Ali shrine AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims gather in Karbala AFP/Getty Images Shia Muslim pilgrims gather for the Arbaeen religious festival Shia Muslim pilgrims gather REUTERS

The pilgrims carry black, green, red and white flags, with the black flag of mourning for Imam Hussein by far the most common. Vast numbers of them decorate permanent brick buildings and temporary tents which are used for praying, eating and sleeping along the three main routes leading to Kerbala. Once pilgrims were lucky if they got rice and bean stew – “there was nothing but muddy water to drink” recalls one early participant – but everything is now highly organised with copious supplies of food, small clinics and even dentists all working for free. The care of pilgrims is regarded as a religious duty.

This year there are more red, white and black Iraqi national flags evident than before, indicating a shift towards greater identification with the Iraqi state by the Shia, traditionally marginalised by the Sunni since Ottoman times and before. When the Shia-dominated government took power in Iraq in 2005 it was the first time the Shia had held power in any country in the Arab world since the Fatimids in Egypt were overthrown by Saladin in the 12th century. It is only now that they have started to look comfortable in their new role.

All religions have their martyrs, but for the Iraqi Shia they come from the present as well as the distant past. Lamp posts fifty yards apart along the 45-mile Najaf-Kerbala road each have a different picture of a soldier or civilian killed by Isis or al-Qaeda. The same is true of other routes to the holy city. Once pilgrims risked death from Isis ambushes, but the roads are more secure. Major General Qais Khalaf, military governor of Kerbala, Najaf and Diwaniyah provinces, says “there was just suicide bombing in this area 18 months ago, when three people were killed.” He believes Isis no longer has the base areas or the level of support it needs to launch big attacks.

The mood of the pilgrimage is one of intense piety and communal solidarity, though Shia clerics keep emphasising that the pilgrimage is dedicated to peace. Asked if the Iraqi security forces’ victory over Isis had an effect on the gathering, Shia clerics said there has been an improvement in morale and self-confidence. “Who does not want more security?” asked Kamil Kadar, a pilgrim taking part in the walk. After 40 years of wars and emergencies, Iraqis are wary of good news, always suspecting that developments will turn sour as they have done so often in the past.

Sayyid Alaa al-Moussawi, a senior Shia cleric who is head of the Office of the Shia Endowment, says that Iraq seems to be entering a period “of greater harmony with its neighbours, notably Saudi Arabia, Turkey and Iran”. Iraqis see this as crucial because it is the combination of domestic insurgency with financial and military support from foreign states that has kept Iraq in a permanent state of war and emergency.

Pilgrims from all over the world head for Najaf, Iranians being much the most numerous, but they also come from Pakistan, India, Azerbaijan, Lebanon and places where the Shia community must be tiny. Many of the pilgrims are teenagers or in their twenties. Asked what the pilgrimage meant to him, one 19-year-old visitor from London said: “It means my whole life to me. It means one small step towards heaven.”

Shia Muslim pilgrims gather in the southern Iraqi city of Kerbala ahead of the Arbaeen religious festival (AFP/Getty) (AFP/Getty Images)

Shia identity is beginning to blend with Iraqi national identity as Iraq looks less like a failed state. There is strong sympathy for Shia struggles in countries like Syria, Lebanon, Bahrain and Yemen. Inside Iraq, there is strong popular backing for the Hashd al-Shaabi or Popular Mobilisation Units as a sort of Shia national guard. Nationalism may be on the rise but it is still trumped by religion as a cause Iraqis will die for. Asked about the Hashd, one observer said: “I don’t think you could have defeated Isis without using ideologically-driven fighters like the Hashd.”

The first time I saw Arbaeen walkers was in April 2003 when US soldiers looked in perplexity and with suspicion at bands of young men, often carrying green palm fronds rather than flags, walking to Kerbala from all over Iraq. This was happening just after the US invasion and the pilgrims were walking unconcernedly past burned out Iraqi tanks and military vehicles. The incoming administrators of the US-led occupation paid the walkers no attention, though they were an early sign of the Shia piety and determination which was to shape the future of the country.