The walls of Um Hussein's living room in Baghdad are hung with the portraits of her missing sons. There are four of them, and each picture frame is decorated with plastic roses and green ribbons as an improvised wreath for the dead.

Um Hussein had six children. Her eldest son was killed by Sunni insurgents in 2005, when they took control of the neighbourhood. Three of her remaining sons were kidnapped by a Shia militia group when they left the neighbourhood to find work. They were never seen again.

She now lives with the rest of her family – a daughter, her last son, Yassir, and half a dozen orphaned grandchildren – in a tiny two-room apartment where the stink of sewage and cooking oil seeps through a thin curtain that separates the kitchen from the living room.

Um Hussein looks to be in her 60s and has one milky white eye. She is often confused and talks ramblingly about the young men in the portraits as if they are alive, then shouts at her daughter to bring tea. She told the Guardian how she had to fight to release Yassir from jail.

Yassir was detained in 2007. For three years she heard nothing of him and assumed he was dead like his brothers. Then one day she took a phone call from an officer who said she could go to visit him if she paid a bribe. She borrowed the money from her neighbour and set off for the prison.

"We waited until they brought him," she said. "His hands and legs were tied in metal chains like a criminal. I didn't know him from the torture. He wasn't my son, he was someone else. I cried: 'Your mother dies for you, my dear son.' I picked dirt from the floor and smacked it on my head. They dragged me out and wouldn't let me see him again.

"I have lost four. I told them I wouldn't lose this one."

Afterwards, the officers called from prison demanding hefty bribes to let him go while telling the family he was being tortured. Um Hussein told the officers she would pay, but they kept asking for more. First it was 1m Iraqi dinars (£560), then 2m, then 5m.

"We had to send [the security men] phone cards so they could call us. They said: 'Your son is being tortured – he will die if you don't pay.' So we paid and paid. What could I do? He is the last I have. I said I would sell myself in the streets, just bring him back to me."

The last call came in December. They demanded a final payment to let him go, by which time, according to Um Hussein and her neighbours, the family had paid 9m Iraqi dinars.

"They asked for 60 hundred-dollar bills. Then they said 30. I begged them and they still said 30. I told them I didn't have it, then they agreed on 20."

She took a taxi with her friend to the agreed meeting point, a mosque on the outskirts of the neighbourhood. The driver went out and handed the money to a man who stood on the corner, a Shia security officer called Rafic.

Yassir was released two days later. Um Hussein didn't know it at the time but a judge had ordered Yassir be released six months earlier. The security men had kept him in detention until his family produced another $2,000 bribe.

Yassir's case is part of a growing body of evidence collected by the Guardian that shows Iraqi state security officers are systematically arresting people on trumped-up charges, torturing them and extorting bribes from their families for their release. Endemic corruption in Iraq has created a new industry in which senior security service officers buy their authority over particular neighbourhoods by bribing politicians, junior officers pay their seniors monthly stipends and everyone gets a return on their investment by extorting money from the families of detainees.

During two trips to the country before and after the US withdrawal from the country on 18 December, the Guardian interviewed 14 detainees and five officers in different branches of the security service in Baghdad. All the detainees said they had had to pay money to be freed, even though most had been acquitted in the courts. Some had been jailed for three days and some, like Yassir, for five years. In three cases, officers changed a detainee's "confession" – often extracted under torture – in return for money. In one case, an officer lost the detainee's documents in return for a bribe and he was released due to lack of evidence. One prisoner we interviewed is still in jail and in the middle of negotiations with officers.

Release does not mean escape. According to one officer we spoke to, men who are released are often detained again because a family that has paid once to get their sons out of a detention centre makes an easy target for more extortion.

We asked Um Hussein if we could meet Yassir. He was hiding, she said, but after speaking to him on the phone he agreed to meet us and arrived an hour later. His young face looked pained. He lifted his shirt to show thick, dark scars on his back. Each scar had a ridge in the middle lined by red tissue with tiny bubbles. At the sight of them Um Hussein turned her head and started to wail.

"The commandos surrounded the area and took us," said Yassir. "There were Americans there who took our pictures. They moved us to the ministry of the interior, where they separated the people who were photographed by the Americans from those who weren't." Being photographed is why he survived at the height of the sectarian killing, Yassir explained.

Even so, Yassir said he was tortured inside the ministry compound for two weeks. "The torture started at midnight and went on until morning," he said. They hung the prisoners up and beat their legs with cables. They also beat the detainees' kidneys. "I still urinate blood," he said. "They wanted me to confess [to fake charges of belonging to al-Qaida] but there was nothing to confess to, so I refused to sign anything."

He was moved to an army base north of Baghdad where he says he was tortured for a further month. The Guardian was supplied with the names of officers and their military units there and in his last detention centre and has checked that these officers exist.

During this time Yassir and his fellow inmates were constantly beaten, he said. "Everyone beat us. When they brought food they beat us. When they moved us they beat us. They beat us so much we stopped feeling.

"The worst was when they hung us for six hours to the window bars with car chains or handcuffs and left us there, sometimes twisting our legs and arms until they dislocated our shoulders."

Every six months, Yassir was moved to a new unit or a new jail where he faced the same torture and interrogation. Finally, four and a half years after his arrest, he was brought in front of a judge. Because he hadn't signed a confession, he said, the judge ordered his release.

Another former detainee, who had paid a bribe to get released, told the Guardian he had confessed under torture to a long list of crimes, including setting improvised explosive devices, assassinations and murders.

"They hung me between two desks," said the man, who did not want to be named, "my legs and arms around a stick." He said they called it the quzi position. Quzi is an Iraqi dish of grilled sheep. "They started beating me with cables. I fainted and they threw water on my face and started beating me again. They finished the beating in the morning and in the evening they started the torture again." On the third day, when he couldn't bear the electric shocks and beating any more, he told them he would confess to anything. "They gave me a paper and I signed it and they said if I changed it in front of the judge I would be tortured again."

After that, the negotiations began, the man said. His confessions carried the death penalty. An officer called his father and the family sold furniture and borrowed money and paid $7,000 (£5,000). Five months later, he was released.

We asked Yassir why he hadn't confessed to anything – he would have paid the same money but saved himself the torture. "I didn't do anything. How could I confess? I was ready to die but not to confess."

The negotiator

Rafic is an officer in one of the most feared security units in Iraq, one of the many commando anti-terrorism units which, at the height of the civil war, had a reputation for being a government-backed death squad.

Rafic looks like a nightclub bouncer: he is tall with a shaven head, a roll-neck sweater and slacks. He shouts into his phone, waves his hands theatrically to passing neighbours and when he laughs he exposes big yellow teeth.

Fellow residents in Rafic's Baghdad neighbourhood treat him with extreme caution. They know he has the power to detain their brothers or cousins for months if not years. But they also need him: he is their negotiator and mediator. They know that when someone is arrested they must go to him to seek help. He will arrange a visit, get a phone smuggled into the prison, reduce the level of torture and arrange for their release. Each service will come at a cost.

When we met him in December he was closing a $5,000 deal with the family of a detainee. He promised them he would send their son blankets and food and assured them the beating and torture would stop. The money was the first of many payments Rafic would receive before the man would be released.

"Corruption has reached the head," said the officer who introduced us to Rafic and who worked with him. "From top to bottom, everyone is rotten. Rafic loves money. His religion and his sect is money, which is very useful because it means at least there is someone to negotiate with, not like in the days of sectarianism when we paid money and they still killed our sons."

Rafic stood outside a small shop where he held his "surgery" every evening, drinking Greek ouzo with his friends and receiving visitors. His scope of business is not limited to detainees but covers anything related to corrupt officialdom, including getting ID cards and passports.

We sat in his car to talk. Like many security officers his eyes twitched and darted like two trapped flies, watching the young men standing nearby, the old men playing backgammon and the man selling tea from a stall by the kerb.

"We are neutral," he said, referring to his commando unit. "We don't do Sunni and Shia any more. We are professional. We detain Shia and Sunni. There is no difference."

How do you make detainees confess? "We hang them from the ceiling and beat them until they are motionless corpses," he said. "Then they confess."

"Look," he added, "the system now is just like under Saddam: walk by the wall, don't go near politics and you can walk with your head high and not fear anything. But if you come close to the throne then the wrath of Allah will fall on you and we have eyes everywhere."

He described the arrest of the Sunni vice-president Tariq al-Hashimi's bodyguards who, it was claimed by the Shia-dominated government, had been paid by Hashimi to assassinate Shia officials. (Hashimi was on a plane heading to Kurdistan when government forces took over the airport, preventing him from leaving. After a standoff, he was allowed to fly but his men where detained.)

"Look what happened to the poor bodyguards of Hashimi, they were tortured for a week. They took them directly to our unit and they were interrogated severely. Even an old general was hanging from the ceiling. Do you know what I mean by hanging?"

In the constricted space of the car he pulled his arms up behind his back.

"They hang him like this. Sometimes they beat them with cables and sticks and sometimes they just leave them hanging from a metal fence for three days. They are torturing them trying to get them to confess to the bombing of the parliament."

They hadn't yet confessed, he said. He described the idea that Hashimi, the vice-president of Iraq, would pay someone $3,000 to assassinate a policeman as "absurd", but said they were also torturing them because they wanted to find the head of Hashimi's political office.

We asked Rafic why, as we had heard, the torture only happened at night. He said the human rights inspectors who visit prisons and detention centres usually came during the day and at night there was a back hall where the torture took place. "Even if the inspector comes [during the day], no one of the detainees dares to speak. A week later we get a letter from the ministry thanking us for our professionalism."

Yassir had said that once an inspector had entered their jail and found him lying on his back, unable to move after a beating in the night. "He asked me what was wrong. I said I was sick and I fell when I was using the toilet. If you say you have been tortured, they will kill you after the inspector leaves."

The investors

On the first floor of an upmarket restaurant in Baghdad, where a waitress with dyed blond hair takes orders, and businessmen and government officials sit in secluded booths smoking water pipes, we met a colonel in the ministry of the interior.

The colonel wore a dark suit and listened to people's requests for help in getting a passport, securing the release of a relative or negotiating the Kafkaesque Iraqi bureaucracy. His portly son, who worked as secretary and bodyguard, sat with a notebook and a pistol.

The colonel explained how the country's endemic corruption had resulted in the industrial scale of extortion of innocent detainees and their families. "Everything is for sale, every post in the government is for sale," he said.

"You pay $300,000 to buy a post as a security chief or military commander of a neighbourhood for a year and you have to get your money back. It's like an investment. But you can never trust anyone in this country – they take your money and a year later they conspire against you and throw you in jail. They are like wolves."

One of his subordinates explained how the officers procured their positions. "The commander of the district buys his post from the politicians or the office of the commander-in-chief. Then the commander rents the post of interrogation officer to his juniors for $10,000 to $15,000 per month, depending on the area. For a Sunni neighbourhood you have to pay a lot of money; for Shia not that much, because most of the arrests take place in the Sunni areas. Then you get your money back from the detainee.

"Sometimes you get really lucky and actually detain someone who is in al-Qaida, and then you can get your full investment in one go: you arrange for him to escape for half a million dollars."

Other officers told similar stories: of extortion and torture by hanging and beating, electricity and nail pulling. "Sometimes they put the prisoners in the 'corner' position. They put them in a corner facing a wall, and everyone passing by had to hit them – a slap, a kick, an insult, just to degrade them and break them," said an officer who worked in a sprawling military base in Baghdad that doubles as a high-security prison.

He was a member of the new guard that came to power post-Saddam Hussein, but he was wearing the sort of moustache that was popular in the Saddam era: thick, well-trimmed on the upper lip with its ends sliding down the edge of the lower lips giving him a sinister, angry look.

The corruption, he said, was partly a way of appeasing officers to win their support, but it was also a way to control them. When they didn't want someone, they could accuse him of corruption.

"I am sitting on a ticking bomb," the man said. "If I don't join the networks of corrupted officers they will threaten to transfer me to one of the frontline divisions. I tell them I don't want to steal, but they can go ahead. I know one day things will change and all the corrupted officers will put on trial."

The days of sectarian killing in Baghdad – the monster that ripped through the city – appear to be over. Sectarianism has taken a new form. Now Sunnis and Shia stick to their separate, walled-off neighbourhoods, the Shia death squads have become government forces and the Sunnis are calling for federalism, which is opposed staunchly by Nouri al-Maliki, the prime minister.

The officer who had introduced us to Rafic said: "We Sunnis have suffered a lot from this government in the past years – detention and kidnapping and extortion."

The only solution, he said, was a federal Iraq in which the Sunni-dominated provinces were separated from the Shia-dominated central state. In the meantime, people were trying to work out ways to protect themselves. "Really bad days are about to come. Since the Americans left, everyone is looking for a gun."

In 2008, during the period of intensive American raids on civilians, they had sold their guns and the Kurds had bought them. "Now," he said, "we are trying to buy them back."

All names in this article have been changed