Barack Obama weighs in to Raymond Davis row as Pakistani anger grows over CIA agent blamed for civilian deaths in Lahore

The western spy of the Hollywood thriller storms through cities of the developing world with blithe abandon – battering vehicles, dodging local police, cutting down enemies with bursts of expertly aimed gunfire.

Real life, it turns out, can be like that too, with one difference: the aftermath is more complicated and costly.

On 27 January, Raymond Davis, a bulky 36-year-old CIA agent with a shock of grey hair, was winding through the chaotic Lahore traffic when he stopped at a red light. A motorbike carrying two men, coming from the opposite direction, swerved in front of his Honda Civic. The pillion passenger was carrying a gun. Davis, a former special forces soldier, whipped out his 9mm semi-automatic Glock pistol and, still behind the wheel, opened fire. Five shots sliced through the windscreen. Muhammad Faheem, a 19-year-old street criminal, fell dead.

CIA agent Raymond Davis.

Davis got out of the car and took aim at the motorbike driver, Faizan Haider, who had started running. Another five shots rang out and Haider fell to the ground, having run 30ft; a postmortem indicated he was hit three times in the front and twice in the back.

Davis walked back to his car, called for help on a military-style radio, then started to photograph the dead men. Anwar Khan watched from his restaurant across the street, amazed at the American's sang-froid. "He was very peaceful and confident. I was wondering how he could be like that after killing two people," he said.

The American rescue squad consisted of a Toyota Land Cruiser, probably manned by fellow CIA agents, that careened through the streets towards Davis. Nearing Mozang Chowk, where the shooting took place, the driver saw the road jammed with onlookers and traffic so he ramped the vehicle over the central reservation and continued at speed against the flow of vehicles. He hit and killed a cosmetics trader riding his motorcycle, Ibad ur Rehman, then pressed on. But Davis was gone.

Apparently panicked by a crowd, the CIA agent had already taken off towards central Lahore, ignoring police who tried to wave him down. At Mozang Chowk, a warden tried to stop the Land Cruiser. Witnesses later told police that one American swung open his door, brandished a rifle and threatened to fire on anyone who got in his way. The Toyota retreated to the US consulate, jettisoning a number of items along the way including 100 bullets, knives, gloves, a blindfold and, oddly, a piece of cloth with an American flag.

About the same time, police caught up with Davis in the crowded bazaar of Old Anarkali, about two miles away, where he was arrested. Outrage started to course through the streets of Lahore.

Since then the "Lahore incident", as Senator John Kerry called it this week, has riveted Pakistan – triggering a media firestorm, plunging the clod-footed government into fresh crisis, and highlighting the deep lack of trust between rival spy services that raises questions about the hunt for al-Qaida in the tribal belt.

A growing cast of characters has become part of the drama. Last week Barack Obama waded in, urging Pakistan to free "our diplomat" under the Vienna convention. "We've been very firm about this being a priority," he said.

Pakistan's foreign minister, who opposed the US position, lost his job over the affair.

And the widow of Faheem killed herself by swallowing rat poison. From her death bed she gave an interview in which she prayed Davis would be brought to justice.

One morning last week a scrum of black-suited lawyers gathered in a Lahore high court to protest against Davis.

"He is enjoying all the facilities, including sharab [wine]!" one lawyer cried. "He should be charged with espionage!" said another. Perhaps Raymond Davis was not even his real name, suggested a lawyer for the Punjab government.

The judge, sitting quietly under a small lamp, listened carefully then issued an order that Davis should be identified by photo at Pakistani border posts to prevent him fleeing the country.

On the streets, many Pakistanis argued Davis should pay for his crime, blood for blood. Giant posters of the three slain men stare down from street corners, the largest being that of Rehman, the 26-year-old motorcyclist mowed down by the American 4x4.

"All Pakistanis agree: Davis should be tried and hanged," said Muhammad Ali, a taxi driver, sipping tea in a dingy café by Mozang Chowk.

Davis awaits his fate inside the sprawling Kot Lakhpat jail. He is refusing to answer questions, but shortly after his arrest he gave police a written statement in which he explained his actions. He had opened fire, he said, in self-defence against robbers who may have seen him coming from an ATM. He opened fire when he saw Faheem, on the back of the bike, cock his pistol.

Police have corroborated some of this. Both men had records for petty crime – "Robberies, small-time muggings, that sort of thing," said a senior official – and a warrant had been issued for Faheem's arrest.

Five mobile phones were found on their bodies, two of which had been stolen. Both men were carrying unlicensed weapons; the photos taken by Davis showed Faheem had indeed drawn his gun.

But then other facts emerged that caused Pakistani prosecutors to pause. They questioned why Davis needed to fire on his assailants 10 times, and why he leapt from his car to shoot one of them, apparently as he fled.

Then, the police say, they discovered that Faheem's gun contained no bullet in the chamber – meaning it could not have been cocked.

Davis has been charged with two counts of murder and one of illegal weapon possession. "He used force that was not commensurate with the threat, that much is clear," said a senior police official.

Pakistanis see the episode as more evidence of imperialistic arrogance. For years the press has been filled with conspiracy-laden speculation about Blackwater – now known as Xe – the American military contractor with a reputation for violent ruthlessness, in their country. Papers have been filled with stories of armed Americans roaming the streets with disdain for the law and innocent life. The US denied the stories. But now Blackwater has been made flesh.

Press coverage zings with unlikely stories about Davis – that he howls in his prison cells when the five-times daily call to prayer rings out; that the CIA plans a "Hollywood-style heist" to spring him; that he is the linchpin of the CIA's drone programme.

One popular suggestion has it that Davis should be swapped for Aafia Siddiqui, the US-educated neuroscientist jailed for 86 years in 2010 on charges of attempting to kill American soldiers and FBI agents in Afghanistan.

The embattled US embassy finds itself fighting this media tide, arguing that Davis should be released under diplomatic immunity.

The tone has veered from aggressive to conciliatory. "We all feel the pain and the anguish of families who have lost loved ones," said Kerry on a visit last week, in which he promised that Davis would be subject to a criminal inquiry if sent home.

A furious debate has erupted over whether Davis qualifies for immunity under the Vienna convention of 1961. The US insists he does, although the story has changed.

Officials initially described Davis as an employee of the Lahore consulate, potentially exposing him to prosecution because consulates fall under a separate treaty. A day later the US hastily announced that Davis was in fact accredited to the main embassy.

"We made a mistake," admitted a senior US official.

What the US has not, or cannot, explain, however, is who Davis really is. Born in Wise, Virginia, he was in the army for 10 years, eventually joining the 3rd Special Forces group until he left in 2003 to become a private contractor. According to a senior ISI official, he worked for Xe. Later, he joined the CIA.

Like many spies, he entered Pakistan on a diplomatic passport, and served for a time in Peshawar before moving to Lahore, where he lived in a large house in an upmarket neighbourhood.

He got his first visa in 2009, and most recently entered the country on 20 January.

When he was arrested a week later, Davis was carrying a 9mm gun and 75 bullets, bolt cutters, a GPS unit, an infrared light, telescope, a digital camera, an air ticket, two mobile phones and a blank cheque.

According to the provincial law minister, Rana Sanaullah, the camera contained photos of "prohibited areas such as installations along the border with India … This is not the work of a diplomat. He was doing espionage and other activities."

Yet US officials insist that Davis is a diplomat, and deserves to be treated as such.

"The Pakistanis gave him a visa, he came on a diplomatic passport, and that's all it takes. There's nothing more to it," said a senior official in Islamabad.

They have a point: spies from around the world, including Pakistan, operate under diplomatic cover.

In 2001 the Indian media reported that an official at the Pakistani embassy in Nepal was found to have 16kg of explosives in his home. The official claimed diplomatic immunity and was expelled from Nepal.

The Davis debacle is another disaster for the Pakistan government, whose handling has been characterised by bungling and division, and highlights the country's pathological relationship with America. "Our people have a funny way of fighting a fire by pouring oil on it," said one dispirited senior official.

How will it end? One solution is a hefty payment to the families of the dead, with the courts quietly dropping the case and Davis being expelled from Pakistan, although this would require an American admission of guilt.

But in a case where fact and fiction collide, nothing is certain.