A few years back, when I was a full-time political journalist, I had a chat with a colleague who had just become a government information officer. It was one of those periods when Fleet Street was taking pot shots at Tony Blair, and I asked my friend how it felt to be embattled. He laughed, saying he had been shocked to find out how little reporters - let alone the public - knew what was going on in Whitehall. "I reckon on any given day you'll be lucky to find out 1%."

As the bizarre arrest of Damian Green is reduced to political squabbling, the bigger picture is being missed. Governments, of whatever hue, are subject to alarmingly little scrutiny. From time to time parliament rises to the occasion, but more often than not has no idea what affairs of state are circumventing it. British journalism is dangerously weak. It veers between hysteria (the "I'm so angry because" school of commentary) and stenography. What is the point of the media if it does not see its primary task as gathering information to hold power to account? Investigative journalism takes time and money. One can count on the fingers of perhaps two hands the serious practitioners, many of whom rely on whistleblowers.

When I was editor of the New Statesman, we ran several stories that rattled the government's cage. My two criteria in deciding to publish were whether a story was entirely accurate, and whether it threatened national security. The most notable case was that of Derek Pasquill, a Foreign Office official. He leaked to Martin Bright, the magazine's excellent political editor, information about the government's approach to militant Islam. I didn't necessarily agree with Pasquill, but I was convinced that his concerns were in the public interest.

Pasquill was harassed, interrogated and charged under the Official Secrets Act. This was a malicious political prosecution designed to save the skins of mandarins and ex-ministers, and we decided to defend him to the hilt - pointing out that ministers had already changed policy to address those points raised by Pasquill, and therefore he had no case to answer. Throughout the case I had the impression that the government had become so arrogant and vengeful against anyone trying to coax out the truth that ministers would invoke all arms of the law to do their bidding.

Faced with bullying, we tried a spot of it ourselves. I went for a one-to-one meeting with a senior minister. I warned him that when the trial began, we would bring forward evidence that would embarrass senior figures in and around government. On the first day of the trial, at the Old Bailey last January, the case was abandoned. Job done, I resigned a month later.

I believe that just as the decision to prosecute Pasquill was political, the decision to drop charges was equally so. There may be more to the case of Damian Green and his Home Office whistleblower, but I doubt it. This too seems an act of brute force. New Labour has a terrible record in this regard, and I believe that the Conservatives would act with as much disdain for the democratic process. Their record under Thatcher and Major suggests little else.

The threat to robust inquiry is perhaps greater now than ever before in our system. Much of British journalism has become supine in the face of intimidation from state organs and from libel laws that encourage the crooked around the world to sue in UK courts to prevent legitimate inquiry.

For some time reporters have complained that editors and proprietors are shying away from difficult stories for fear of "getting into trouble". The fear with the Green case is that civil servants and politicians will now do so as well.

• John Kampfner is chief executive of Index on Censorship indexoncensorship.org