Germany's spies seriously violated the country's laws multiple times, according to a secret report from its federal data protection commissioner Andrea Voßhoff.

The legal analysis, leaked to Netzpolitik, was made in July 2015 following a visit by data protection officials to Bad Aibling in southern Germany, in the wake of Edward Snowden's revelations about surveillance activities there. Bad Aibling is jointly run by Germany's intelligence agency, the Bundesnachrichtendienst (BND), and the NSA.

As well as listing 18 serious legal violations, and filing 12 formal complaints—the German data watchdog's most severe legal instrument—the secret report said that the BND created seven databases without the appropriate legal approval. As a result, commissioner Voßhoff said that all seven databases should be deleted, and could not be used again.

Significantly, one of the illegal databases used the XKeyscore software, sometimes called the NSA's Google. As Ars reported last year, it was known that the BND had a copy of this program, but the Netzpolitik leak appears to provide details of the huge scale on which it was used:

For the SIGINT [signal intelligence] collection, i.e. as so-called front-end system, XKEYSCORE—using freely definable and linkable selectors [keywords]—scans […] the entire Internet traffic worldwide, i.e. all meta and content data contained in Internet traffic, and saves selected Internet traffic data (e-mails, chats, content from public social media, media, as well as non-public—i.e. not visible to the normal user—messages in Web forums, etc.) and hence all persons appearing in this Internet traffic (sender, receiver, Web forum member, member of social networks, etc.). In real time, XKEYSCORE makes these Internet traffic data—attributed to its users—readable and analysable for an agent.

One consequence of scanning the entire Internet is that information about many innocent individuals was gathered, according to the report:

Because of its […] systematic conception, XKEYSCORE—indisputedly—collects […] also a great number of personal data of irreproachable persons. The BND is not capable of substantiating their number […]. In one case I checked, the ratio was 1:15, i.e. for one target person, personal data of fifteen irreproachable persons were collected and stored, which were—indisputedly—not required by the BND to fulfill its tasks […].

Voßhoff said that the BND not only broke German law by using XKeyscore, but also because it sent the information it gathered to the US: "The content and metadata collected via XKEYSCORE are transferred to the NSA, following an automatic clearing of information falling under the G-10 law (G-10 assessment). These transmissions are additional severe violations of fundamental rights." The data transfer was on a huge scale—some 14 million items every day. As Ars reported a year ago, handing over data to the NSA was part of the deal for the BND to obtain the XKeyscore software.

The new leak provides details about how the BND tried to sanitise the information it sent to the NSA, using a data filtering system called DAFIS, which was supposed to remove all data originating from German citizens and individuals as required by article 10 of the German constitution. However, the data protection commissioner concluded that filtering wasn't perfect which meant that transfers had breached German law.

The report reveals that the BND built another database that was even bigger than the one generated using XKeyscore, known as VERAS 6, which stored all metadata of every communication for three months. As a result, the BND was once again breaking the law by storing information about innocent members of the public: "By diverting and collecting all metadata of all traffic on a communication line, the BND also stores and uses metadata of communication traffic by irreproachable persons which are not necessary to fulfill the BND’s mission. This means metadata of irreproachable persons is also stored in VERAS 6 and used for metadata analysis."

According to the secret report, the BND was not limited to a few "hops" of metadata to connect individuals, but could extend the graph indefinitely: "All persons having a connection to a directly relevant person, or if their metadata are stored because of a geographical perspective are indirectly relevant for the BND. The connection to a directly relevant person can be established over any amount of hops. VERAS 6 does not have a restriction. "

The construction of social graphs in this way will also be possible in the UK thanks to the Investigatory Powers Bill, which will require ISPs to store everyone's metadata. The similarity between the UK and German government's approach to spying in fact runs deeper.

Just as the so-called Snoopers' Charter aims to put surveillance activities by UK spies on a firmer legal footing, so the German government has proposed a new law that would effectively legalise the activities criticised so harshly by Voßhoff in the leaked report. According to Netzpolitik, the legislative package is scheduled to be adopted this year and will probably come into effect at the start of 2017.

Meanwhile, peers in the House of Lords will return on Monday afternoon to continue to scrutinise the IPB at committee stage following summer recess.