DOHUK, Iraq — For as long as anyone alive can remember, political loyalty has been imposed in Iraq. Repressive regimes forced the people here to join parties and causes. One day they were Communists, the next Ba’athists, then the KDP, the PUK, the State of Law party or the Mutahidoun would snap them up. Now many are with ISIS, and one of the key questions for those who hope to roll back the self-declared “caliphate” is whether the local Sunni Muslims who decided to collaborate with the onslaught of the Islamic State will continue to support it, or change sides once again.

For the Yazidis, followers of an ancient religion regarded by ISIS as heretical and therefore punishable by death, the issue of Sunni loyalties is a matter of survival for their families and their culture. As ISIS rolled into their villages with the tacit and sometimes enthusiastic support of Sunnis they thought they knew, the Yazidis were driven from their farms and homes, the men were slaughtered by the hundreds, and their wives and daughters were abducted, imprisoned and sold into slavery.

It’s often said that sectarian hatreds are endemic and inevitable in the Middle East. But for centuries, Muslims and Yazidis have lived side by side as friends, neighbors and economic partners in the area around Mount Sinjar, which rises from the plains of northern Iraq. Most Yazidis had a kreef, a Muslim who is like a brother. They drank tea together and attended each other’s funerals and weddings.

The diversity and harmony of the Sinjar district was common in the province of Ninewa (or Nineveh), where Yazidis, Christians, Kurds, Turkmens and Sunni and Shia Arabs have lived together for centuries. The provincial capital, Mosul, was one of the great cosmopolitan trading cities on the caravan routes from Asia to Europe, and even when the caravans disappeared, it maintained traditions of openness.

There have been tensions from time to time, certainly, and some groups dwindled as younger generations searched for better futures in bigger cities or abroad. But each group retained its identity and its traditions—until June when ISIS conquered Mosul, which is the second biggest city in all Iraq, and began a campaign to exile or exterminate non-Muslims living in the regions it called the Islamic State.

Multiple interviews with Yazidis from Sinjar reveal that some—though not all—of the Muslims around them either acted as bystanders or participated actively in ISIS attacks against them earlier this month. And many Yazidis say that because of this they can never return to live in their former homes among those Muslim neighbors: “There will always be something between us now,” said one woman from Sinjar.

The Yazidis, who are living as refugees today, say it wasn’t just Arab Muslims from Sinjar who joined the Islamic State. There were also Kurdish Muslims, or “Muslim Krmanj” as locals know them. (Kermanji is a Kurdish dialect used in Sinjar.) “If we go back they will do this again in maybe three years, maybe more, but they will do it again,” says a Yazidi man from the Snuny area north of the mountain.

The anger is palpable. But there is also disappointment. “We expected this,” said a man from Khana Sor. “Even though we are friends, we have never fully trusted the Muslims.” He said that while Muslim girls were allowed to come to Yazidi houses, he would not send his daughter to a Muslim house out of fear someone would try to convert her.

Politics also played a role in the calculus of the Arab Muslims who supported ISIS. Since the American-led invasion that toppled dictator Saddam Hussein in 2003, Sinjar has been a disputed territory between the central government in Baghdad and the Kurdistan Regional Government in Erbil. But over the years the Kurds, mainly through the Kurdistan Democratic Party (KDP), have become by far the dominant force in the district thanks to patronage politics. Yazidis, recognized as ethnic Kurds, helped build that base. One reason the Kurdish politicians were so anxious to make inroads: the region is thought to have major untapped oil reserves that could help the Kurds gain energy independence, and then, eventually, political independence.

Yazidis from Sinjar say that there was resentment among Sunni Arabs about the Yazidi contribution to the rise of the Kurdish influence and control. This resentment and loss of power may have pushed some of them to join ISIS.

One example is Muhammed Asel, kreef to the Yazidi and a Muslim Arab sheikh from a village on the north side of the mountain. Yazidis from Sinjar report that ISIS has named him the wali, or mayor, of Snuny, a region north of the mountain.

But it’s also true that many Muslims stayed behind just because they could. Convinced that it was unlikely ISIS would kill them, since they were Sunni Muslims already, they decided that staying and protecting their property was better than going to live in a refugee camp while everything they left behind was looted.

The local Sunnis may also have believed that they had nowhere else to go. The neighboring government of the Kurdish region, which took in the Yazidis, turned Arab Muslims away because of suspected ISIS sympathies. Now those Sunnis who stayed behind must find ways to survive, which means continued cooperation with ISIS.

Property—and greed—is also an issue. “People are different, some are more human than others,” said an older Yazidi man from Khana Sor, arguing that the reasons some stayed and some left had to do with their individual personalities.

When I asked groups of Yazidis about whether their Muslim neighbors collaborated with ISIS , one man said “it was all of them”; another responded, “but we can’t say all, it was only some,” another person has a story about a collaborator, and yet another about a kreef who helped someone escape.

Yazidis say that the Muslims from Sinjar who did play the ISIS game had started doing so long before ISIS arrived on their doorsteps. They participated first as the ISIS “mukhabarat,” or intelligence service, passing important information about the Yazidis to ISIS before the assault began. Two Yazidis say that Muslims hosted ISIS members in their homes to stage the attacks.

Muslims in several villages near the border have had longstanding ties to rebels in Syria, including ISIS, because they used to smuggle weapons from Iraq to Syria to support them in their fight against Bashar al Assad.

On the south side of the mountain a survivor of one of the ISIS massacres, who stayed alive by hiding underneath the bodies of dead friends and relatives, knew the ringleader of the group that carried out the slaughter. Another commonly mentioned collaborator is a famous Yazidi singer who converted to Islam years ago. Locals say he led many of the attacks south of the mountain in Sinjar district and encouraged killing Yazidis instead of releasing them. “The two most dangerous types of people are poor who become rich and those who convert to Islam,” observed one man from Snuny.

However there are also many reports of kreef and other Sunni friends helping Yazidis. When one Yazidi family from Khana Sor ran to the home of their kreef, he gave them traditional Arab clothes and helped them to escape.

One woman from Hardan, Wahida Rashu, recalled that one of her relatives’ kreef helped her immediate family escape through Rabiaa from Hardan. Another Yazidi man from south of the mountain near Tel Banat told me that his kreef is protecting his sheep.

Many Yazidis say that the powerful Sunni Arab Shammar tribe of Rabiaa district—historic friends and economic partners of Yazidis in Sinjar—did not participate widely in the attacks. Indeed, several of them gave Yazidis their cars to escape.

What all this tells us is why it will be so difficult to push back the forces of ISIS. The vital cooperation and support of the Sunni tribes in the territories they’ve conquered is deeply complicated, and there is no single approach that might enlist the sympathies of them all. Rocket and bomb attacks will not win loyalty nor, for that matter, impose it. That will require an understanding of the villages, the tribes, the sheikhs and the shepherds. Perhaps it is no wonder that President Barack Obama said Thursday he has no strategy yet for defeating ISIS.