The events surrounding the police in the past 10 days are all too familiar - heavy-handed treatment of innocent bystanders, followed by botched cover-ups. Meanwhile, Bob Quick's inadvertent exposure of highly sensitive information follows a tradition of lost discs, while the government demands to collect our personal data to protect us. The only comic side to the recent sad events is that they were all caught on camera, many of which were owned by the police themselves. Rough justice or what?

That makes Quick risking his briefing papers for an anti-terrorist operation being photographed quite extraordinary. And the behaviour of the police officer attacking Ian Tomlinson from behind even more so. What could they have presumed would be the reaction of anyone seeing those pictures? They must have known cameras were everywhere.

They may have felt safe from criticism because of Section 76 of the Counter-Terrorism Act, which came into force on 16 February. That makes it an offence to photograph any police officer or member of the armed services in ways that could aid terrorism. This problematic legislation risks discouraging an ever more precious weapon in holding police accountable when the government gives them powers to treat peaceful demonstrators as a threat not only to order, but to the state itself.

How else to explain the trashing of the Climate Camp earlier this month, just the latest heavy-handed response to environmental protests and other causes. These include even the green-wellied phalanxes on the Countryside March. Many of these questionable police actions are taken under anti-terrorism legislation, fuelled by uncritical ministers seemingly unaware of the perverse outcome. They are undermining the social contract between police and the public that is central to good order: policing by consent.

The risk of terrorism has added emphasis to the police's long-held sense that any protest threatens law and order. It means their planning for demos not only includes provision for reasonable behaviour, it is often based on worst-case scenarios. That too often becomes a self-fulfilling expectation. Police announcing in advance to protesters at G20 that "we're up for it" primes officers in riot gear to expect trouble. If they don't find it in front of them, they go looking for it - indeed, cause it.

That's the still unlearnt lesson of so many marches far back in my memory. Take the Grosvenor Square anti-Vietnam demonstration in 1968. Police horses charged a peaceful crowd that had filled the square. Why? I later learnt from an officer on the US embassy steps that anarchists at the front were throwing ball bearings. Thousands behind them had no knowledge of this, but endured sudden police efforts to clear the square. I was caught in a group squeezed by a cordon of officers arm in arm - not out of the square but against some railings. A superintendent was caught with us. I suggested we'd be crushed unless they released one end of the cordon and let us leave. They did.

In the 1970s, the Special Patrol Group were used at protests that had got out of hand. Cooped up for hours in vans, they made trouble worse by lashing out at whoever was around. Violence was blamed on demonstrators. Sound familiar?

The 1980s riots, coupled with heavy policing of the miners' strike, became a virtual civil war between sections of the public and police. In battles such as Orgreave and Wapping, police saw themselves as "piggy in the middle" between Thatcher's wish to crush the unions and belligerent strikers. But like the G20, some officers, who behaved normally on their own at home, lost it when in acting in crowds. The police became the enemy.

Despite years of public order training, when tempers fray, police professionalism is sorely tested. I saw that close up at the Notting Hill carnival in 1993. Things had improved after senior officers realised that using riot police in packed crowds could be counterproductive. Officers were told to be friendly and non-confrontational. It worked until it was dark and time to go. Crowds found the shortest route to Ladbroke Grove tube blocked by a barrier and two officers telling them gruffly to take the long way. They didn't explain to tired and drunk revellers the route was saved for ambulances. So some threw bottles.

Cue the riot police to join what was a minor altercation. Hundreds piled into the nearly empty Portobello Road with shields and batons. They had spent the hot August day in a school with their uniforms on, watching Rambo. Their helmets and shields offered ready targets for the bottle throwers - and anonymity in case an odd photographer snapped them.

The trouble with using riot police to act peacefully is that it's a contradiction in terms. Moreover, the police focus on worst-case scenarios not only makes them more likely, but they also fail to distinguish between the small core of troublemakers who want confrontation and innocent bystanders, such as Ian Tomlinson seems to have been. It plays into the hands of those who wish to show the heavy hand of the state, including terrorists.

Police are still doing this by "kettling", confining all who happen to be near the centre of the demonstration for many long hours. It may limit local damage, but it causes much longer damage to their reputation. And often the young caught in the kettle will later rise to positions of power on the bench and in parliament.

This heavy-handedness is especially counterproductive. Not only does it contradict the recent new key performance indicator of increasing public confidence, it also makes it less likely that people will provide useful intelligence against potential terrorism. Intelligence-led policing is the new mantra. But intelligence involves more than taking pictures of everyone at a demo and collecting our emails, texts and travel movements on an insecure database. It requires understanding, sensitivity and discretion, all of which go out the window when the red mist descends.

• Roger Graef is an executive producer of The Truth About Crime, to be broadcast on BBC1 in May

• Nick Cohen is away