ERBIL, Iraq – It was one of the most shocking events in one of the most brutal periods in Iraq’s history. In late 2005, two years after the U.S.-led invasion toppled Saddam Hussein, U.S. soldiers raided a police building in Baghdad and found 168 prisoners in horrific conditions.

Many were malnourished. Some had been beaten.

The discovery of the secret prison exposed a world of kidnappings and assassinations. Behind these operations was an unofficial Interior Ministry organisation called the Special Investigations Directorate, according to U.S. and Iraqi security officials at the time.

The body was run by militia commanders from the Badr Organisation, a pro-Iran, Shi’ite political movement that today plays a major role in Baghdad’s war against Islamic State, the Sunni militant group.

Washington pressured the Iraqi government to investigate the prison. But the findings of Baghdad’s investigation – a probe derided by some of its own committee members at the time as a whitewash – were never released.

The U.S. military conducted its own investigation. But rather than publish its findings, it chose to lobby Iraqi officials in quiet for fear of damaging Iraq’s fragile political setup, according to several current and former U.S. military officials and diplomats.

Both reports remain unpublished. Reuters has reviewed them, as well as other U.S. documents from the past decade.

The documents show how Washington, seeking to defeat Sunni jihadists and stabilise Iraq, has consistently overlooked excesses by Shi’ite militias sponsored by the Iraqi government. The administrations of George W. Bush and Barack Obama have both worked with Badr and its powerful leader, Hadi al-Amiri, whom many Sunnis continue to accuse of human rights abuses.

Washington’s policy of expediency has achieved some of its short-term aims. But in allowing the Shi’ite militias to run amok against their Sunni foes, Washington has fueled the Shia-Sunni sectarian divide that is tearing Iraq apart.

The decade-old U.S. investigation of the secret prison implicates officials and political groups in a wave of sectarian killings that helped ignite a civil war. It also draws worrying parallels to the U.S. government’s muted response today to alleged abuses committed in the name of fighting Islamic State.

Those accused of running the secret prison or of helping cover up its existence include the current head of the Iraqi judiciary, Midhat Mahmoud, Transport Minister, Bayan Jabr, and a long revered Badr commander popularly referred to as Engineer Ahmed.

“Special Investigations Directorate personnel illegally detained, tortured and murdered Iraqi citizens,” the U.S. report states. “Iraqi government officials failed to take action to stop the crimes.”

“We do not violate the human rights.” Muen Kadhimi, the Badr Organisation

The report says U.S. investigators faced a “lack of government cooperation, reluctance of witnesses to come forward and the perception of official complicity.”

Judge Mahmoud declined to comment for this story. A former colleague close to him said Mahmoud knew about the secret prison’s existence but did not know what went on there: “He cannot be held responsible for every judge’s behaviour.”

Transport Minister Jabr did not respond to Reuters’ queries. Jabr has previously stated publicly that no wrongdoing occurred at the prison.

A senior Badr official told Reuters that the prison allegations were part of a smear campaign by terrorists. He called for the international media to focus on Islamic State, which has carried out suicide bombings and executed prisoners.

U.S. officials acknowledge the role that Shi’ite militias such as Badr play in fighting Islamic State. As the Hashid Shaabi, or Popular Mobilisation Forces, the militias helped Baghdad defend the country against the Sunni jihadist group when Iraqi military and police divisions deserted en masse in 2014.

Since then, the militias have continued to attack Islamic State, which has declared a Caliphate across swathes of Iraq and Syria. Islamic State, also known as Daesh, routinely executes citizens who speak against it, kidnaps people, buys and sells women and children, and uses rape as a weapon.

American ambassador Stuart Jones told Iraqi state television in April this year “that the Hashid Shaabi is part of the Iraqi fighting forces which are defeating Daesh today.”

But Sunnis in areas freed from Islamic State control say the Shi’ite militias have been guilty of their own excesses, including looting, abductions and murder. At least 718 Sunnis in Salahuddin province have been abducted by fighters from Shi’ite militias since April 2015, according to several security officers, a provincial council member and tribal leaders. Only 289 have been freed, most after paying ransoms.

Some former and current U.S. officials say Washington needs to stop downplaying abuses by the Shi’ite militias.

Robert Ford, a former U.S. diplomat who served as the U.S. embassy’s political officer between 2004 and 2006, believes the U.S. government’s decision not to punish those behind the secret prison set a damaging precedent. “A few people were transferred elsewhere,” he said. “That’s not a punishment. You are supposed to scare them into not doing it.”

Ten years ago, Ford said, the militias were armed groups with political agendas, or the armed wings of political factions. “Now … the Prime Minister’s office has called them an official institution, and they receive resources directly from the state as well as a degree of political legitimacy.”

A Badr official, Muen al-Kadhimi, dismissed recent allegations of kidnapping, looting and killing. “We do not violate the human rights and we should not forget the inhumane ways practised by the enemy of the Iraqi people,” Kadhimi told Reuters.

The Iraqi government conceded there has been a problem with kidnapping around Iraq, even in Baghdad, sometimes by men in security uniforms. “The Iraqi government is working hard to fight this,” said Saad Hadithi, a spokesman for Prime Minister Haider al-Abadi. He blamed “gangs” for the attacks, but said the state had “no concrete evidence of who is behind it.”

The U.S. embassy in Iraq and the State Department’s new counter-terrorism envoy, Brett McGurk, did not respond to requests for comment.

THE MESS

The Badr group spent years in exile in Iran. Its parent organisation, the Supreme Council for the Islamic Revolution in Iraq (ISCI), was the most powerful Shi’ite political force in Iraq.

After Saddam’s fall, Washington hoped ISCI and Badr would be reliable partners for the security forces, which Badr members joined in large numbers. But despite claims that they had demobilised after their return to Iraq, Badr’s fighters did not disarm, U.S. army intelligence officers say. Instead, they began to assassinate former Iraqi officers, influential Baath party members and civil servants.

Colonel Derek Harvey, a retired intelligence officer, told Reuters that the U.S. military detained Badr assassination teams possessing target lists of Sunni officers and pilots in 2003 and 2004 but did not hold them. Harvey said his superiors told him that “this stuff had to play itself out” – implying that revenge attacks by returning Shi’ite groups were to be expected. He also said Badr and ISCI offered intelligence and advice to U.S. officials on how to navigate Iraqi politics.

After Shi’ite religious parties swept to victory in elections in 2005, Badr and ISCI were given control of the Interior Ministry. The U.S. embassy publicly backed the move. But James Jeffrey, the top U.S. diplomat at the time and later ambassador to Iraq, was alarmed when Bayan Jabr, a Badr ally, became minister. “Bayan Jabr was the biggest mistake I made,” Jeffrey told Reuters. “His file was terrible.”

Jabr appointed Badr members to senior Interior Ministry posts. They created the covert Special Intelligence Directorate, which current and former U.S. officials believe coordinated the killing of former Saddam-era officials. Within months, Sunni politicians reported a sharp increase in the abduction of Sunni men. Some Sunnis blamed men in police uniforms. Corpses began to turn up around Baghdad.