Shocking footage shot by police, accompanied by their own critical commentary, shows how their officers monitored campaigners and the media – and demanded personal information – at last August's climate camp demonstration in Kent guardian.co.uk/Jason N. Parkinson

Police are targeting thousands of political campaigners in surveillance operations and storing their details on a database for at least seven years, an investigation by the Guardian can reveal.

Photographs, names and video ­footage of people attending protests are ­routinely obtained by surveillance units and stored on an "intelligence system". The ­Metropolitan police, which has ­pioneered surveillance at demonstrations and advises other forces on the tactic, stores details of protesters on Crimint, the general database used daily by all police staff to catalogue criminal intelligence. It lists campaigners by name, allowing police to search which demonstrations or political meetings individuals have attended.

Disclosures through the Freedom of Information Act, court testimony, an interview with a senior Met officer and police surveillance footage obtained by the Guardian have ­established that ­private information about activists ­gathered through surveillance is being stored without the knowledge of the people monitored.

Police surveillance teams are also ­targeting journalists who cover demonstrations, and are believed to have ­monitored members of the press during at least eight protests over the last year.

The Guardian has found:

• Activists "seen on a regular basis" as well as those deemed on the "periphery" of demonstrations are included on the police databases, regardless of whether they have been convicted or arrested.

• Names, political associations and photographs of protesters from across the political spectrum – from campaigners against the third runway at Heathrow to anti-war activists – are catalogued.

• Police forces are exchanging information about pro­testers stored on their intelligence systems, enabling officers from different forces to search which political events an individual has attended.

Lawyers said tonight they expect the Guardian's investigation to form the basis of a legal challenge against the use of police surveillance tactics.

Liberty, the human rights group, is challenging the police surveillance tactics in a judicial review at the court of appeal. But police appear not to have disclosed to the court that they were transferring private details of campaigners to a database.

Corinna Ferguson, Liberty's legal officer, said: "A searchable database containing photographs of people who are not even suspected of criminal activity may well violate privacy rights under article 8 of the Human Rights Act. It is particularly worrying if peaceful protesters are being singled out for surveillance."

Police surveillance footage from the climate camp demonstration in Kent last August, obtained by the Guardian, reveals how journalists are monitored as well as the often clumsy nature of the ­surveillance.

It shows police are interested in the names, clothing, whereabouts, and personal details of protesters and journalists. Three members of an ITV news crew, a Sky News cameraman and several photo­graphers were among members of the press monitored as they left the camp. Later in the day journalists at a protest against the Kingsnorth coal-fired power station, were followed by ­surveillance officers to a McDonald's restaurant. Police filmed them as they used the restaurant's Wi-Fi ­connection to file their material.

Kent police have already apologised after official complaints about the incident and intrusive stop and searches of journalists covering the demonstration.

The National Union of Journalists has been assured ­that members of the press were not being ­targeted after it took concerns to the Home Office and senior police ­officers. The union documented at least eight ­protests since last March where its ­members were "routinely" photographed and filmed by police. Several journalists said police officers they had never met knew their names. "We have put this to police and the Home Office several times but they have always denied the practice or sought to avoid answering the question," said Jeremy Dear, the union's general secretary. "With this evidence there is no credibility in doing so any longer."

Police have not disclosed the number of activists on the database. But court testimony by surveillance officers has confirmed the existence of a large intelligence system which, according to one officer, contains "thousands" of campaigners.

Overt surveillance by police forward intelligence teams (Fits) or evidence ­gatherers (EGs) is designed to record potential criminal activity and gather ­useful intelligence. Pioneered by the Met's public order branch in the late 1990s, the technique is used regularly across the country. Surveillance officers use ­"spotter cards" to identify activists. Police have always denied surveillance is conducted for the purposes of storing information on a database.

Information released by Scotland Yard under the Freedom of Information Act has revealed that while raw surveillance material is stored in a warehouse, material on certain individuals "is added to a corporate intelligence database". Scotland Yard's disclosure, in response to questions from NUJ lawyers , stated "generally, records are retained for seven years".

Superintendent David Hartshorn, from the Met's public order branch, conceded law-abiding campaigners were being added to the database. He said individuals on the system included people convicted or suspected of public order offences.

But he added "people we have seen on a regular basis involved but may not have been charged or arrested" were also stored on the database. He added that the data was reviewed every year. "In relation to what we can keep on databases, we are governed quite strictly on that. Obviously you've got the Data Protection Act but also, in terms of intelligence, we have to justify what we are able to keep."