As demonstrations go, it was more of a lighthearted affair than a threat to the nation.

About 600 climate change campaigners had gathered outside the Drax power station in North Yorkshire. They had chosen to demonstrate there because the huge plant is the UK's biggest emitter of carbon. The protesters were mainly families with young children, accompanied by clowns, cyclists, baton twirlers and, according to some reports, a giant ostrich puppet.

It was not completely without incident. Two protesters climbed a lighting pylon at the edge of the site and four others broke through the fence. About 30 others were arrested for public order offences.

Under the heading of "not much a fight, more like a festival", the Guardian reported that the predicted battle between the police and activists wanting to close the plant down had not materialised.

It was the type of demonstration which has been going on for decades in Britain. But the police appear to have had another, completely different view of the 2006 protest.

After the demonstration, the first in what has become an annual gathering known as Climate Camp, North Yorkshire police conducted a review along with government officials. Internal papers obtained by the Guardian show they called it "the first time domestic extremism took place against national infrastructure in the county".

The term "domestic extremism" is now common currency within the police. It is a phrase which shapes how forces seek to control demonstrations. It has led to the personal details and photographs of a substantial number of protesters being stored on secret police databases around the country. There is no official or legal definition of the term. Instead, the police have made a vague stab at what they think it means. Senior officers describe domestic extremists as individuals or groups "that carry out criminal acts of direct action in furtherance of a campaign. These people and activities usually seek to prevent something from happening or to change legislation or domestic policy, but attempt to do so outside of the normal democratic process." They say they are mostly associated with single issues and suggest the majority of protesters are never considered extremists.

Police insist they are just monitoring the minority who could damage property or commit aggravated trespass, causing significant disruption to lawful businesses. Activists respond by claiming this is an excuse that gives police the licence to carry out widespread surveillance of whole organisations that are a legitimate part of the democratic process.

They also warn that the categorisation carries echoes of the cold war, when the security services monitored constitutional campaigns such as the Campaign for Nuclear Disarmament and the Anti-Apartheid Movement because alleged subversives or communists were said to be active within them, although they said the organisations themselves were not subversive.

The domestic extremist term was coined by police involved in tackling criminals involved in animal rights groups sometime between 2001 and 2004. Many of these activists were prepared to resort to violence to promote their aims, most notoriously digging up a grandmother's grave.

The police were successful in jailing many of the animal rights campaigners who were committing crimes. However, there are fears the police's domestic extremism apparatus, which evolved to counter sometimes violent criminals, is now looking for new targets to justify both its budgets and its existence.

There are three little-known organisations at the heart of this apparatus. They work in tandem under the direction of Anton Setchell, who is national co-ordinator for domestic extremism for the Association of Chief Police Officers (Acpo).

The main branch is the National Public Order Intelligence Unit (NPOIU), essentially a giant database of protest groups and protesters in the country.

Housed at a secret location in London, its purpose is "to gather, assess, analyse and disseminate intelligence and information relating to criminal activities in the United Kingdom where there is a threat of crime or to public order which arises from domestic extremism or protest activity".

Police in England and Wales collect intelligence on individuals and then feed it to the NPOIU which, Setchell said, "can read across" all the forces' intelligence and deliver back to them "coherent" assessments.

Setchell said the "fair proportion" of the intelligence comes from Special Branch officers and police who monitor and photograph demonstrations.

Sensitive information from informants in protest groups and covert intercepts are handled by a section of the NPOIU called the Confidential Intelligence Unit.

The NPOIU database consists of entries indexed by descriptions of people, nicknames or pseudonyms.

Originally it was confined to animal rights groups, but was expanded in 1999 to "include all forms of domestic extremism, criminality and public disorder associated with cause-led groups". It contains some information supplied by companies that hire private investigators to spy on protesters, sometimes by infiltration.

Setchell argued that there were robust safeguards to protect the human rights of individuals on the database. He said it was possible that protesters with no criminal record were on the databases, but police would have to give a justified reason.

"Just because you have no criminal record does not mean that you are not of interest to the police," he said. "Everyone who has got a criminal record did not have one once."

The second part of Acpo's triumvirate, the National Extremism Tactical Co-ordination Unit (Netcu), helps police forces, companies, universities and other bodies that are on the receiving end of protest campaigns.

Netcu's job is to give "security advice, risk assessments and information that can minimise disruption and keep their employees safe". Its head, Superintendent Steve Pearl, says his 16-strong unit works with police forces across the country, keeps detailed files on protest groups, rather than individuals, and liaises with thousands of companies in aviation, energy, research, farming and retail.

Netcu was set up in Huntingdon, Cambridgeshire in 2004 by the Home Office which, Pearl said, was "getting really pressurised by big business – pharmaceuticals in particular, and the banks – that they were not able to go about their lawful business because of the extreme criminal behaviour of some people within the animal rights movement."

Pearl denied the unit was engaged in mission creep but admitted that environmental protesters had now been brought "more on their radar" as they had been "shutting down airports, and shutting down coal-fired power stations, more recently stopping coal trains, hijacking coal trains and ships in the river Medway."

The third leg of the trio, the National Domestic Extremism Team, was set up in 2005 and consists of detectives who help police forces around the UK.

Initially, the team focused on animal rights activists, but has fanned out to look at any crimes "linked to single issue-type causes and campaigns", Setchell said.

The team draws on intelligence from the NPOIU database and, the Guardian has learned, is located on the seventh floor of 10 Victoria Street in central London, a building previously occupied by the Department of Trade and Industry.