To the inhabitants of Kandahar City, Ahmed Wali Karzai is a symbol of everything wrong with their home, an emblem of the murky nexus of warlords and criminal syndicates controlling southern Afghanistan's largest city.

In the words of some residents, the half-brother of Afghanistan's president is accused of being a "warlord, a terrorist", a narcotics trafficker, and a contract monopolist. Others won't even mention his name. "I can't tell you anything about this. I'm too scared. Someone might kill me," one resident said.

Pressure on President Hamid Karzai to sack his brother from the provincial council he chairs has led nowhere. Now, as military operations get under way in Kandahar's rural districts – the "cornerstone" of Nato's counterinsurgency campaign – Nato officials hope that they can co-opt Ahmed Wali and the handful of powerbrokers dominating the political landscape here.

"It's very difficult to untangle but what's really fuelling the insurgency is groups being disenfranchised, feeling oppressed by the institutions of state and criminal syndicates," said Mark Sedwill, Nato's top civilian official in Afghanistan. The message was repeated in more than a dozen interviews with Afghan and Nato officials, private citizens, analysts and local journalists. The biggest problem is not the Taliban, it's the gangster oligarchs in charge. Or as Sedwill put it: "I'm not sure whether I'm watching Godfather part 2 or Godfather part 3."

Forget the Corleones, Kandahar's political order revolves around two families: the Sherzai and the Karzai. An uneasy rivalry exists between them, symbolic of a wider tribal jostling between Sherzai's Barakzai tribe and the Popalzai group to which the Karzai family belong.

Both families are allegedly linked to the narco-mafia, criminal gangs and government corruption, although Nato has been unable to provide President Karzai with hard evidence of his brother's alleged involvement in illegal rackets. "Like any mafia organisation the guys who really matter are not the ones you have any evidence against," Mr Sedwill said. Mr Ahmed Wali Karzai denies accusations of corruption

The absence of evidence has done nothing to assuage Ahmed Wali's reputation for squashing opponents and accumulating power. Allegations resurfaced in the New York Times this week, with Western officials claiming he was involved in a host of criminal activities, including money laundering, racketeering and electoral fraud. They also claimed he paid insurgents not to attack his business interests.

Other sources said he oversees a number of armed groups. Residents say that these private armies may be behind the assassinations of provincial officials such as Sitara Achekzai and Yunus Hosseini, whose deaths have been blamed on the Taliban. And as thousands of US troops pour into Kandahar, Ahmed Wali has apparently been seizing land he thinks Nato may want to rent. He already rents a compound outside the city to the CIA and US Special Forces, and helps the agency run a paramilitary strike force. Gul Agha Sherzai, meanwhile, is a former governor of the province. His brother, General Abdul Razziq Sherzai, is one of the two main beneficiaries of contracts on Kandahar airfield, one of Nato's mega-bases. Shunted out of office because of Western concerns about his involvement in the narcotics trade, he has emerged as one of Afghanistan's most important power-brokers.

In his office behind rows of blast barriers, the provincial attorney general, Mohammas Ismael Zia, told The Independent that he often has to drop cases under pressure from members of Kandahar's ruling elite. "Many people call me from parliament, from the governor's office, from the provincial council office, saying 'Release this man, drop that case,'" he said. "I am a weak man. If I don't accept their demands maybe I will get killed. They are threatening me."

"In Kandahar every criminal has a supporter and the supporter wants him released from custody," he said. "There are many warlords in Kandahar City. Don't write their names. If I don't accept their demands, they can make many problems for me. They could kill me or remove me from this job. So sometimes I ignore the rules."

Also complicit in the problem are the police. Tales of policemen springing kidnaps, selling arms to the insurgents and preying on civilians are commonplace. One interviewee told the story of a man who refused to hand over his dog to a police commander, at which point the officers threw him in jail on suspicion of being an insurgent. "The worst people, the addicts, the thieves, the drunkards are in the police," Haji Abdul Karim, a tribal elder said. "The best way for bad men to make money is to join the police." This collapse of the rule of law is one factor driving people towards the Taliban, with their reputation for impartial albeit brutal justice. Balanced against the reputations of the warlord rulers in Kandahar City are those of Taliban commanders like Kaka Abdul Khaliq, who operates in the village of Pashmul, about 25 miles west. Residents said he was well liked and respected. "He treats villagers well because he understands them," said Haji Mohammad Zahir, who knows him.

As gun battles rage in the outlying districts this summer, Nato wants to wage a political campaign in Kandahar City to remove the culture of impunity and provide jobs and security; it's not the Taliban but the power-brokers and crime syndicates there who are to blame for this situation.

Officials want to take a three-pronged approach. By holding large numbers of shuras, or councils, and rolling services out into the districts, they hope to foster better, more inclusive governance. They want to empower the mayor and governor to mitigate the influence of "malign actors". And they also want to push Afghan officials and elders to resolve tribal tensions over what some tribes see as an unfair distribution of resources. The most visible example of this is in the dilapidated mansion outside Kandahar airfield owned by General Abdul Razziq Sherzai. It's a symbol of his grip on the market for contracts from Kandahar airfield – a major source of cash. "The dis-equilibrium is clearly driving some of the insurgency," one official said. Nato has begun an informal review of the contracts it has awarded.

Establishing the political will at the heart of the Afghan government is also critical to repairing Kandahar's political fabric. "What it boils down to is getting down there with President Karzai, the key ministers, the governor, the provincial council chairman, the other big figures and saying 'Right, we've got to sort out this political context because otherwise everything else we do, security, governance, development, is frankly just anaesthetising it,'" Sedwill said.