On Thursday 18 July activists will gather outside the Royal Courts of Justice to protest the enduring secrecy that surrounds the decade-old death of a retiring civil servant. Dr David Kelly was Britain's foremost authority on biological weapons, and perhaps our leading expert on Iraqi WMD, yet as this protest reminds us, his sudden death met with no inquest and no evidence was heard under oath. But despite this basic injustice, and despite my own long years of doubt, I will not be on the courtroom steps that morning. I have broken faith, and this is my confession.

The core group of Kelly campaigners has included doctors, surgeons, solicitors, a psychologist and at least one QC, although this has not prevented some commentators from labelling them as cranks, or – to use the specific pejorative – conspiracy theorists. Yet the morning Kelly's body was found, we were all conspiracy theorists: bar perhaps a few spooks, the whole nation from the prime minister down had no idea what had transpired on Harrowdown Hill. Kelly had been the only member of the Whitehall monolith to tell a journalist what hundreds of his fellow civil servants knew full well: that Downing Street's argument for invading Iraq was founded on deliberate dishonesty. In a haunting final email, he complained he was beset by "dark actors, playing games", and hours later he was dead.

The Hutton inquiry, in its brazenly pro-government account of Kelly's death, did nothing to assuage our distrust, and so for a while I too joined that disaffected legion of disbelievers who spent their nights trawling chat rooms and internet forums. There were a lot of us about. Conspiracy theories were inevitable.

Firstly there was shocking act of Kelly's suicide itself. The vast majority of people who kill themselves have a diagnosable mental illness or a history of prior attempts, and almost half of those who do not leave a note or some other indication of clear intent. When a man of sound mind abruptly decides to kill himself, especially in such an extraordinary and unlikely way, it is as mysterious as it is tragic. Then there is Kelly's obscured relationship with western intelligence agencies, and the ferocious political storm that engulfed him, But most profoundly of all there is the war itself.

War, whether we support it or not, begs for a narrative that can furnish us with heroes and villains. For the majority of British people, who saw the Iraq war as an unnecessary disaster, the villain was not Saddam but Tony Blair, and, casting about for a hero, we fell on a mild-mannered boffin, a man who spoke truth to power at great personal cost, the whistle-blower who told us "the government probably knew it was wrong".

As my investigation proceeded, this narrative unravelled entirely. I followed Kelly from his childhood in a smallpox-stricken Welsh valleys town to the university where he met his first military microbiologist (and where secretive smallpox work was carried out). I tracked him from his Oxford institute, whose staff sometimes experimented on the Porton ranges, to his tenure at Porton Down itself, from where he made his final and fatal leap into the world of intelligence. The man I discovered was not a meek civil servant but a deliberate, hard-edged expert who never once departed from the official line he was given.

Kelly was a man of secrets, and he kept them all, from the mysterious eco-terrorists supposedly responsible for the anthrax parcels of Operation Dark Harvest to Britain's tacit co-operation with apartheid South Africa's biological weapons programme. And not just South Africa. It was via Pretoria I obtained documents showing Kelly had also escorted two Iraqi microbiologists around his military lab, shortly after the Iran-Iraq war, and from another country again that I unearthed confirmation of the biological exports to Baghdad that our government hid from the Scott inquiry. Kelly never spoke out about any of it.

Similarly, the hue and cry about Iraqi WMD did not spring up out of nothing in 2002. It was a show that had been running for years, and as a chief inspector of the UN special commission, Kelly had always been centre stage. It was during this time that Kelly's original sponsor in British intelligence, the late Brian Jones, began to lose sight of him.

Jones had brought Kelly aboard the Defence Intelligence Staff (DIS) to help with the debriefing of a Soviet defector. By the late 90s, according to Jones, Kelly had become a man who drifted in and out of the analytical DIS with unspecified clearance and an unclear agenda, governed by a vague and deepening relationship with the Secret Intelligence Service. Underlying this was Kelly's role at Unscom, supposedly a multinational disarmament body, but in reality a tool that western intelligence used to enforce sanctions, encourage regime change and provide the rationale for military intervention. None of this appears to have troubled Kelly. He raised no objection to Operation Desert Fox in 1998, just as he raised none to Operation Iraqi Freedom in 2003.

In reality, Kelly was something of a dark actor himself. When Jones tried to get the language of the "September dossier" toned down, Kelly was one of those who opposed him. When it came to the laughable notion of mobile Iraqi biolabs Kelly told the BBC's Susan Watts, among others, he was "90% certain" they existed. Only after his unexplained deportation from Kuwait, when he was handcuffed, searched, and had his belongings confiscated, did he and his colleagues drop the pretence that they would find WMD. Off the record, he began briefing journalists that their existence was unlikely. To Andrew Gilligan, he said the government had suspected this all along.

I don't think Kelly ever gave an unauthorised interview in his entire life. Jones believed his post-Kuwait briefings were intended to manage public expectations after the invasion, and to make sure it was the government, not British intelligence, which got the blame for confecting claims about Iraqi WMD. But the political fallout was cataclysmic. Downing Street went on an unprecedented offensive, and Kelly found rattled senior spooks were turning against him. Just as politicians and mandarins were traducing his professional reputation, counter-intelligence was set to tear his private life apart. Shortly before his death his top secret clearance was revoked. It would have heralded the most intrusive investigation imaginable, and to save himself and his family from the indignity, he walked up Harrowdown Hill and reduced his security risk to zero.

It is a tale bereft of heroes, and so it displeases every camp. Some of those who believe Kelly was murdered have called me an intelligence plant. Conversely, pro-war "rationalists" have said I am driven by an anti-Blair vendetta. Meanwhile it appears that intervention in the Middle East is imminent yet again, and I have no doubt we will see more David Kellys in the British press. Meticulous? Of course. Cautious and reserved? Naturally. On message? Always, and without fail, until the cracks begin to show. I fear that as long as this country values theatre over debate, we will never be short of dark actors.