Police chiefs face a legal challenge over their policy of keeping secret files on thousands of political activists.

A police monitoring group and campaigners have started legal action to curb a clandestine database that records its subjects' political activities.

Police say the database is necessary to keep track of campaigners they deem to be actual or potential "domestic extremists".

They say they need to identify hardcore protesters who have used, or are about to use, criminal means to promote their political beliefs.

However, evidence gathered by the Guardian shows that many listed on the national domestic extremism database have never committed a crime.

Documents revealed that police spied on one campaigner while he was at the Glastonbury festival and, in another case, kept a detailed log of an 88-year-old pensioner with no criminal record when he attended 55 demonstrations over a four-year period.

In legal papers submitted to the Metropolitan police and home secretary Theresa May, lawyers for the police monitoring group Netpol and environmental campaigner Lilianna Etkind say police are collecting and storing information on "numerous individuals involved in peaceful public protest" in breach of the Human Rights Act.

The surveillance of campaigners has been controversial following revelations about the conduct of undercover officers who were sent to spy on political groups. Intelligence gathered by the undercover officers has been stored on the database, along with information from sources such as informants, covert intercepts, websites of campaigns, and officers who openly film and monitor demonstrations.

The number of individuals on the database of the national domestic extremism unit run by Scotland Yard fluctuates. The latest available total is 7,500, although it is known to have topped 9,000.

Some campaigners have obtained copies of their own entries under data protection laws. John Catt, the 88-year-old pensioner from Brighton who campaigns on peace and human rights issues, found on his file descriptions of his appearance at protests such as "clean-shaven", the slogans on his clothes and even his habit of sketching demonstrations.

He won a ruling at the court of appeal that ordered police to delete the file they were unlawfully keeping on him.

Guy Taylor, 46, discovered that the police had recorded how he and other campaigners set up a stall at Glastonbury festival and sold leftwing publications and merchandise. He has a conviction for spray-painting anti-war slogans in 1991.

Val Swain, of Netpol, said: "There is a database culture within modern policing that has been allowed to run wild. Yet the law does not effectively regulate the ways in which this data is collected, how it is kept and used or who it is shared with. This has to change."

Rosa Curling, the solicitor from Leigh Day legal firm who has sought the judicial review, said: "The database is not established under legislation and there exists no sufficiently clear, precise and accessible policy setting out its scope and purpose."

She has sent a legal letter to Scotland Yard and the home secretary as a prelude to launching the judicial review.

Scotland Yard confirmed that it had received the letter but declined to comment further.