Momentous changes are underway in European intelligence, propelled by new technology and a political push for integration.

And without finally having an open and inclusive public conversation about them, we risk losing the democratic legitimacy of these transformations.

Student or retired? Then this plan is for you.

Paris struck by major attack in 2015

The evolution of government surveillance is bold, multi-faceted, and confusing. Agencies across the continent are deploying an avalanche of new technologies, notably machine learning, to both advance new capabilities such as biometric surveillance and master long-standing challenges like information overload.

New institutions of intelligence cooperation and comprehensive data-sharing arrangements are implemented at multiple levels, across jurisdictions, and with greater-than-ever corporate involvement.

At the same time, the EU is itself advancing an ambitious new surveillance agenda and is investing heavily in security research.

And while national and European courts still grapple with questions of legality, territoriality, and privacy brought about by the previous round of intelligence laws, parliamentarians across Europe are already debating new intelligence bills that aim to grant agencies even greater intrusive surveillance powers.

New technology is driving the transformation

Whereas the proliferation of data mushroomed with the widespread integration of digital devices into people's everyday lives, advanced surveillance equipment allows intelligence agencies to collect that data at unprecedented scale and in real time.

Artificial intelligence can then help them get a grip on the sheer volume, velocity, and complexity of modern data analysis.

It remains entirely unclear, however, whether existing legal frameworks cover the use of such technologies.

Automated and machine-learned cues for offensive cyber operations might enhance the services' effectiveness, but they also require much more attention to unresolved questions around rights infringements, operational accountability, and oversight.

Using facial recognition software, the city of Nice, one of many trial locations in Europe, may be able to identify attendants of a carnival.

But should democracies collect the most personal of personal data this aggressively, let alone act upon it?

Naturally, new means of data collection from a growing pool of sources - and the convergence of surveillance technology in intelligence, law enforcement, and the military - prompt ever-closer working relations across sectors and countries in Europe.

However, they also should - and regrettably often do not - occasion new ways of structuring liability and oversight.

Consider, for instance, the Counter Terrorism Group's operative platform in The Hague where 30 different intelligence services submit data into a giant database, without clear rules for who is liable for wrongly imputed or corrupt data that can have real adverse human consequences.

Oversight

While there are good reasons for intense intelligence sharing, it is problematic when each service, as is currently the case, is accountable only to its national oversight bodies.

Not only do these institutions typically review but a fraction of their respective agencies' international activities, even the best national oversight structure would be unable to rein in joint intelligence action on its own.

Some might say that intelligence, once known as the last bastion of national sovereignty, has become an arena of collusive delegation.

The response has to be multilateral oversight however tricky this may be to administer.

Similarly, at the EU level, national datasets are being operationalised into new "cross-information analysis platforms".

Recent examples of this include the Counter-Terrorism Register at Eurojust, the Inspectr project, and the European Search Portal.

With the latter, personal profiles based on fingerprints and facial images stored in the free-travel Schengen Zone security systems and data from joint police agencies Europol and Interpol can be accessed by an increasingly large amount of officials across the EU.

According to the European Commission, more than €1bn are earmarked for these and other massive 'interoperability' projects until 2027.

By rendering vast amounts of personal data even more accessible for law enforcement, border, and intelligence agencies across jurisdictions, Europe further erodes already waned demarcation lines between operational agencies and those that are collecting information about potential threats.

Worrying

This is a worrying development.

In some instances, notably Austria and Latvia, police and intelligence functions are already performed by a single agency. Granting inter-jurisdictional access and pushing for greater interoperability will do away with important firewalls that history has wisely dictated in some countries.

Currently, our open societies - the ones that are meant to benefit from this development - have far too little information, let alone say, on this agenda. Moreover, our oversight and data protection bodies have yet to catch up on the technological revolution.

Their audit and review mechanisms, for example, are everything but fully synced, automated, and comprehensive. This invites abuse and does little to close ever-growing accountability gaps.

If we want to protect fundamental rights and align European surveillance practices with our democratic principles, we need to start a meaningful public dialogue about the momentous changes that are taking place.

With the stakes being so high and the potential for mutual learning so great, it is striking that people in the agencies and oversight bodies, government, civil society, business, and academia rarely engage in regular, open, and inclusive conversation on these matters.

Given the complexity of security policy in a rapidly changing world, it may seem preferable to bury our heads in the sand and let the ones inside the ring of secrecy run the show.

This, however, is a luxury that neither the people outside nor the people within the corridors of power can afford.