This week has seen thousands of Kurds fleeing the city of Kirkuk in the face of the advance of the Iraqi army. The scenes conjure up memories of Saddam Hussein’s campaigns against the Kurds.

The reversal of Kurdish fortunes in this city come just weeks after Kurds voted overwhelmingly for an independent Kurdistan.

In 2003, as the governorate coordinator of Kirkuk, I witnessed the struggle for control of the surrounding province in the aftermath of Saddam’s overthrow. Kirkuk is home to different communities including Arabs, Kurds, Turkmen and Christians. It also has oil. And a troubled past in which the Ba’ath party had sought to ensure an Arab majority in the province by expelling Kurds and Turkmen, and importing Arabs from the south.

The Kurds were seeking to redress 35 years of ethnic cleansing, encouraging those expelled from Kirkuk to return while at the same time pressuring “new” Arabs brought up from the south to leave. And they were pushing to restore the pre-1976 borders of the province. All of this was aimed at ensuring an overwhelming Kurdish majority in the province so that they could annex it – and its oil – to Kurdistan.

But the drive to make Kirkuk part of Iraqi Kurdistan was rejected by Arabs and Turkmen.

I was working for the Coalition Provisional Authority, and one of its greatest challenges was managing the competition between the region’s rival factions. In December 2003, a demonstration was held that was billed as a celebration for the arrest of Saddam. Kurdish flags were everywhere, along with some American flags. Banners declared that Kirkuk should be part of Kurdistan; and that Kurds who had been deported from the province by the former regime should come back.

Arab and Turkmen leaders felt compelled to react. A few days later, a crowd gathered in a square in Kirkuk. Many Iraqi flags were visible, with the takbir (“God is great” in Arabic) supposedly in Saddam’s handwriting between the stars of the Ba’ath party. There were also blue Turkmen flags. Banners declared Arabs and Turkmen were one, and called for “one country, one people, no ethnic federalism”. Some provocateurs rushed towards the offices of the Kurdish political parties. Shots were fired and four people were killed.

For months, we had been beseeching Baghdad to grant Kirkuk a special status. We argued that the “Kirkuk issue” could derail the national discussion on Iraq’s new constitution. There was real potential for conflict within the province that we feared could spill over into other parts of the country and encourage the involvement of external actors. Some form of special status could defer the determination of Kirkuk’s final standing for five years – to provide the time and space to resolve the issues, and strengthen local leadership.

We believed special status had the support of the local population – and would stop Kirkuk being a political football between Baghdad and the Kurds. But despite our arguments, the coalition did not grant such status.

Article 140 of the Iraqi constitution set out a process for resolving the status of those territories disputed between Baghdad and Irbil (the capital of Iraqi Kurdistan). But it was never implemented.

Over the intervening years, the Kurds exerted greater control over the province. Kirkuk became a stronghold for the Patriotic Union of Kurdistan party, which selected the governor.

When the Iraqi security forces fled in the face of Isis in 2014 it was the Kurds, with support from the US-led coalition, who fought back and pushed them out of Kirkuk.

Masoud Barzani, president of the Kurdish regional government, sensed the Kurds were in their strongest-ever position, holding territory and weapons, and decided to push ahead with a referendum on full-blown Kurdish independence, including Kirkuk. He believed he could improve the Kurds’ bargaining position for negotiations with Baghdad over separation, and therefore ignored the objections of Baghdad, Turkey, Iran, the US, the UK and the UN. The vote took place on 25 September.

In forcing the referendum, Barzani sought to rally Kurds around the flag – hoping to deflect attention away from the financial crisis in Kurdistan, accusations of corruption and criticism of his staying on as president beyond his legal term.

But Kurdish over-reach has been met by a strong response from Iraq’s prime minister, Haider al-Abadi. He deployed federal forces to push the peshmerga out of Kirkuk. An Arab has been appointed by Baghdad as the acting governor. And Arabs and Turkmen are celebrating their victory.

Long-term stability in Kirkuk, however, requires a political settlement. Now, more than ever, there is a need to negotiate the future of the disputed territories. It is time to revisit the idea of a special status for Kirkuk, with power-sharing between the different communities. The future of Kirkuk should be determined by politics and compromise – not by force. • Emma Sky is author of The Unravelling: High Hopes and Missed Opportunities in Iraq, and a former governorate coordinator of Kirkuk