For the past two years, a tight-lipped and little talked about unit within the Metropolitan Police has been conducting blanket surveillance of British citizens' public social media conversations. Following an unintentional leak and a detailed investigation, we are finally able to see some of the capabilities of this 17-man team—some of which are truly alarming.

The PRISM scandal engulfing US and UK intelligence agencies has blown the debate wide open over what privacy means in the digital age and whether the Internet risks becoming a kind of Stasi 2.0. The extent of the UK's involvement in this type of mass surveillance—which already appears exhaustive—shows just what a potential intelligence goldmine social media data can be.

But the monitoring of our online trail goes beyond the eavesdroppers in GCHQ.

For the past two years, a secretive unit in the Metropolitan Police has been developing the tools for blanket surveillance of the public's social media conversations. Operating 24 hours a day, seven days a week, a staff of 17 officers in the National Domestic Extremism Unit (NDEU) has been scanning the public's tweets, YouTube videos, Facebook profiles, and anything else UK citizens post in the public online sphere.

The intelligence-gathering technique—sometimes known as Social Media Intelligence (Socmint)—has been used in conjunction with an alarming array of sophisticated analytical tools.

"Sentiment analysis" that can determine your mood, "horizon scanning" that tries to pre-empt disorder and crime, facial recognition software that can track down individuals, geo-location that is able to pinpoint your whereabouts, and profiling that can map who you are and what circles you move in. All innovative techniques used in the private sector, and all adapted for law enforcement and surveillance.

As the head of open source intelligence in the Met, Umut Ertogral revealed in May during what he intended to be a private presentation at an Australian security conference [according to a couple of Met sources, conference organizers "forgot" to tell the audience that the talk was off the record]:

"[Social media] almost acts like CCTV on the ground for us. Just like the private sector use it for marketing and branding, we've developed something to listen in and see what the public are thinking."

The unit has been building and honing this new style of surveillance ever since the 2011 London riots.

While the NSA's PRISM program collects data that is supposed to be hidden from the outside world, this form of open source intelligence eavesdrops on the data you haven't made private—be it intentional or through ignorance. But several privacy groups and think tanks—including Big Brother Watch, Demos, and Privacy International—have voiced concerns that the Met's use of Socmint lacks the proper legislative oversight to prevent abuses from occurring.

"The issue of legal protection and privacy sits at the heart of all this," explains Carl Miller, research director and co-founder of the Centre for the Analysis of Social Media, which has been leading the research into Socmint. "What we really need is a clear and enabling framework, both legislation and regulation, which can explicitly inform people when and why the police can collect this kind of information."

Current legislation intended to protect the public from abuses, like the Regulation of Investigatory Powers Act (RIPA), were passed at a time when Facebook and Twitter didn't exist.

Surveillance operations often require a ministerial sign-off or permission from a superior, but it is unclear whether targeting of public social media data requires the same level of oversight, as head of research at Privacy International Eric King points out.

"Millions of British citizens share billions of pieces of information about their lives with social networking sites every day," he explains. "While RIPA authorizations are required for most methods of offline surveillance, the police are refusing to come clean about what checks and safeguards—if any—are in place to ensure that surveillance of online activities stays lawful and proportionate."

Challenged with these concerns, the Met still refused to go into detail about the unit but told Wired.co.uk that its use of Socmint was necessary "to protect communities."

"Police have a duty to uphold the law and prevent and detect crime," said a spokesperson from the Specialist Crime and Operations Desk. "Online channels will attract those intent on committing crime, engaged in gang activity, or communicating with rival gangs to fuel tension and threaten violence."

The Met also argued its intelligence was "publicly available material which is readily accessible to all using the Internet." But some are not so convinced this private/public dividing line is quite so clear-cut.

Dr. Daniel Trottier, a researcher in Social and Digital Media at Westminster University, argues that the sophistication of the tools able to analyse this data means we should see our public social media output in a different light.

"The perception with this kind of intelligence is that it's in the public domain, so it's no different from, say, searching through newspaper articles," he elaborates. "But this analysis shows a lack of familiarity with the technology involved and the extent to which it can identify and analyze people.

"There's a psychologist in Cambridge, for example, who showed how with just a few statements from social media profiles, one is able to reasonably determine a user's sexual orientation.

"Now, whether or not these kinds of predictions are accurate is beside the point—it's the fact that the predictions are taking place at all and are taken seriously that's important. If your online conversations flag you up as a potentially troublesome individual, regardless of whether you are or not, you will still end up being blackballed as such."

Some say they have had first-hand experience of this kind of targeted profiling, and it has pushed them to go off the grid almost altogether.

"It's got to the stage where I will only use a public telephone or meet someone face to face if I want to discuss something sensitive," explains Janie Mac, a legal observer for the Occupy LSX movement. "We are all very aware that our accounts are being monitored. We've moved our social network activity to make it more private and we've moved away from traditional social sites for our online meetings and discussions."

It is suspected that protesters and political activists are bearing the brunt of the Met's Socmint surveillance program.

On June 26, The Guardian reported that the very same unit had a "secret database" that had labelled some 9,000 individuals—many from political groups—as "domestic extremists." It adds to the growing number of questionable surveillance tactics used by the police. What is particularly troublesome is that these abuses occurred even with the apparent existence of proper legislation and oversight—something the snooping of social media data currently does not have.

"With anything to do with surveillance, we must look at whether it is necessary, proportionate, and in the public interest," concludes Emma Carr, deputy director of Big Brother Watch.

"In some cases it certainly will be, but there has to be a clear framework that's unambiguous and consistently applied. Only then can the public start to feel comfortable that it's being used proportionately and in their interest."

This story originally appeared on Wired UK.