British troops came close to capturing al-Qaida's top commander and the occupation forces' most wanted target in Iraq – but the operation collapsed after the only surveillance helicopter ordered to monitor him ran out of fuel and had to return to base, secret military intelligence logs suggest.

The astonishing blunder in March 2005 allowed Abu Musab al-Zarqawi – a Jordanian associate of Osama bin Laden with a $25m reward on his head – an extra 15 months to expand al-Qaida's operations throughout Iraq, bringing the country close to civil war. His fundamentalist Sunni supporters were behind some of the worst atrocities aimed at Iraq's Shia majority population as well as countless attacks on US and Iraqi government forces. Their bombing of a sacred golden-domed Shia shrine in Samarra in February 2006 led to a wave of revenge killings that lasted for a year and a half.

He was eventually located by the Americans in a house north of Baghdad in June 2006 and killed with his family by a US air strike.

His narrow escape from British troops and a unit of British special forces emerges from the secret military intelligence logs examined by the Guardian. They report that on 17 March 2005 the G3 cell of army intelligence at British brigade headquarters in Basra heard that Zarqawi was travelling south on route 6 from Amarah to Basra. They informed Danish forces at 2.15pm. The Danes played a junior role in the coalition under overall British command in south-eastern Iraq.

Half an hour later, the report says, a Lynx helicopter spotted a suspicious car that had stopped 7.5 miles (12km) south of al-Qurna and about 60 miles north of Basra. Al-Qurna is a dusty flyblown Mesopotamian town that is claimed, inappropriately in view of its present condition, as the site of the biblical Garden of Eden.

US forces in Iraq invariably sent helicopters in pairs. The report says the helicopter maintained "top cover" for 15minutes but then had to return to the UK-run Shaiba logistics base to refuel.

"As a result the area of interest was unobserved for between 20 and 30 min," the report adds. By then British troops from Corunna company of 1st Battalion the Duke of Wellington's Regiment had rushed in and set up an inner and outer cordon around the area. British special forces and an American "arresting officer" were brought in.

Having lost their helicopter cover the forces were reduced to random searching. A Shia mosque was raided but Iraqi civilians were the only people in it. A second building was searched but also only contained civilians. The report ends with: "At 22.14 the search was concluded."

Unlike many other reports in the logs, this one makes no comment on the source of the intelligence and its reliability. The British may have doubted it since Amara is in an overwhelmingly Shia area and perhaps an unlikely place for Zarqawi to be. Perhaps they suspected the Americans were inflating his importance. Regardless, the British did take the report seriously enough to send dozens of troops to capture the suspect.

The near-miss was hugely expensive, if Zarqawi was indeed in the suspicious car. He had first come to international attention when his gunmen kidnapped Ken Bigley, a British contractor working on reconstruction projects, and two American colleagues in Baghdad in September 2004. They were shown on their knees in videos and beheaded two days later after the authorities refused to negotiate their release.

This was a minor atrocity compared to what al-Qaida was to do in Iraq in 2005 and 2006. Led by Osama bin Laden, the movement was not active in Iraq before the US-led invasion in March 2003, even though the Bush administration rejected CIA intelligence to that effect and publicly insisted Saddam Hussein and al-Qaida were linked.

Eliza Manningham-Buller, the head of Britain's MI5 at the time, told the Iraq inquiry last month that by toppling Saddam, George Bush and Tony Blair opened the door to al-Qaida. "Arguably we gave Osama bin Laden his Iraqi jihad so that he was able to move into Iraq in a way that he wasn't before," she said.

Abu Musab al-Zarqawi became friends with Bin Laden when they were both resisting the Soviet occupation of Afghanistan in the late 1980s . Arrested in his native Jordan in 1992, Zarqawi was convicted of trying to overhrew the monarchy and establish an Islamic caliphate. On release from a five-year prison term he went back to political insurgency, setting up his own movement called al-Tawhid wal Jihad. After the US-led invasion of Iraq he aligned himself with al-Qaida and was recognised as its leader in Iraq with the title "emir of al-Qaida in the country of two rivers".

The war logs suggest he made a slow start. Secular Iraqi Sunni nationalists had started their resistance to the Americans within weeks of Saddam Hussein being toppled, and Zarqawi moved in several months afterwards.

In February 2004 the logs report the Italians as having information about plans for a suicide attack in Nasiriya by "possibly an al-Qaida cell" of two men, one Saudi and the other Lebanese wuth false documents. In August 2004 there is a report of a possible al-Qaida safe house in Ramadi , and in December in Baghdad the Americans mounted an unsuccessful raid to capture a "suspected Wahhabi supporter of al-Qaida".

In July 2005 the logs contain a chilling letter of warning to all Iraqis working for the Americans. Addressed to "the traitors", it says: "The tawheed and jehad movement is warning you, if you do not stop your work with the American forces you will be dead and this is a final warning to you from the leader of al-Qaida in Mesopotamia. If you will ever enter an American base or work with them any more, you and all with you will be killed. God is the Greatest, signed by Abo Mosa'ab al-Zarqawi, leader of al-Qaida in Mesopotamia."

After the British fiasco in al-Qurnah the first menton of Zarqawi's potential whereabouts in the logs comes on 22 September 2005 when he is reported to be moving between two Sunni villages in the al-Jazira area north of Ramadi. An intelligence report claims there are 600 foreign fighters loyal to al-Qaida in the area with plans to attack the US base in Ramadi, known as Camp Blue Diamond.

In February 2006 Zarqawi mounted his biggest coup. Armed men blew up the golden dome of the al-Askari mosque, sacred to Shias, in the largely Sunni town of Samarra. The attacks enraged Shias across Iraq, set off a wave of reprisal attacks on Sunnis and Sunni buidings. Sectarian killings that lasted almost a year and a half forced hundreds of thousands of Iraqis to abandon their homes, with many fleeing abroad.

No group took responsibility but the Americans accused al-Qaida and Zarqawi. They redoubled their efforts to find him and on June 7 located him in a house near Baquba. Air strikes were called in and the house was flattened.

The logs are full of "threat reports" about possible attacks on the Green Zone in Baghdad and other US targets to avenge his death. In the event, though, the assassination passed off without a major response.