MI5 used a secret meeting to persuade judges at the UK’s top intelligence and security court not to disclose any information on sensitive databases holding highly intrusive records about the population.

Classified documents released on 28 July 2016 show that MI5 invited six senior judges from the Investigatory Powers Tribunal (IPT) – the body that oversees the UK’s intelligence agencies – to a series of briefings and lunch.

The aim was to persuade the judges to keep data held on mass databases, known as bulk personal datasets, secret from people who filed complaints to the tribunal.

The document emerged during a four-day legal challenge by Privacy International over the lack of safeguards over the use of bulk personal datasets by MI5, MI6 and GCHQ.

The datasets hold highly sensitive information on the population, including location and travel history, internet and mobile phone use, and financial information.

The decision meant that the IPT could not carry out its role of overseeing the work of the intelligence agencies effectively, and was unable to identify errors or mistakes in the data, the group said.

Setting up the briefing MI5 spent months in 2007 struggling to get as many of the members of the tribunal together on the same day for a briefing on the importance of the bulk datasets at MI5 headquarters at Thames House, London, by the service’s then deputy director general Andrew Parker. A heavily redacted file note dated 15 November 2007 revealed: “The main objective was to educate the tribunal members in our data-handling techniques, including our gathering and storing of bulk data, and to seek their agreement that searching of this ‘reference data’ was not necessary or proportionate when responding to an IPT complaint.” The document reveals that the president of the tribunal, John Mummery, along with John McInnes, Peter Scott, Richard Gaskill, Robert Seabrook, tribunal register Veronica Selio, and secretary Sue Cavanagh, visited MI5 in September 2007. They were given five and a half hours of briefings, including lunch with MI5’s senior officials and lawyers. None of the people named has any involvement with Privacy International’s complaint.

MI5 used databases to identify targets The Security Service briefed the tribunal members on the intelligence value of data held in MI5’s collection of bulk databases, described as ‘reference data’. Officials explained they could use the datasets to identify targets from fragments of information and then track their activities. They gave three practical examples to show how the datasets benefit national security, and demonstrated the steps they take to ensure necessity and proportionality in the use of the databases, according to the document.

Lunchtime proposal Over lunch, Parker, now director general of MI5, together with the security service’s legal advisers and other officials, put forward their key suggestion – how the IPT should approach complaints that required querying the agency’s secret bulk databases. MI5 said it would search its databases and other records in the event of a complaint, if the subject had been a target for investigation. It there was a hit, it would provide confirmation to the IPT that the record existed, why it was made and what action it had taken in connection with the investigation. “It the complaint does NOT produce a ‘hit’, we will inform the IPT that we do not hold a record on the complainant, though they may well appear in one of our reference databases,” officials told the group. “In such cases, we will not check our reference data general for any mention of the complainant’s name.”

IPT members accepted MI5’s ‘deal’ The officials suggested that this approach would adequately meet the IPT’s need to be able to judge whether the service had engaged in any unauthorised conduct, or had acted improperly towards the complainant. In return for MI5’s offer, IPT members appeared to accept MI5’s view that it would not be necessary to disclose any data from bulk personal datasets during complaint hearings. “IPT members do not question our assertion that research of ‘reference data’ in response to a complaint was neither necessary nor proportionate,” the memo records. Caroline Wilson Palow, general counsel of Privacy International, said the visit by the IPT members to MI5 was improper in itself, and the document shows that the IPT was failing to act as a neutral party in adjudications. “A fundamental requirement of any court is to treat each part as fairly and neutrally as possible. What this document shows is a possible failure of that neutrality,” she said.

MI5 had no safeguards over bulk personal databases until 2010 MI5 itself realised there was a problem with its oversight of bulk personal datasets, but not until 2010, the tribunal heard on 28 July 2016. In February 2016, Robert Hannigan, now the director of GCHQ, published a report on behalf of the Cabinet Office that addressed the lack of oversight mechanisms in place. Ben Jaffey, representing Privacy International, said: “Prior to 2010, there was no oversight of bulk personal databases at all. The security service accepted internally that it was a problem, as per the Hannigan Report; Mr Hannigan did recommend oversight.”

Tribunal ‘failed in oversight role’ The court heard that the security agencies were able to identify individuals even in anonymised data sets by matching them with data sets containing personally identifiable information, despite assertions otherwise. Jaffey told the court that it is foolish to suppose data that does not specifically include a claimant’s name cannot be tracked back to that individual. Financial datasets containing bank accounts numbers and sort codes can easily be “de-anonymised”, he said. There was also lack of oversight over the way UK intelligence agencies share data with overseas intelligence agencies. “The safeguards are manifestly inadequate. The tribunal wouldn’t know about the fact of the sharing,” he said.

IPT 'failed to plug the gap' in its own oversight Jaffey said the court had effectively prevented itself from acting as a crucial constitutional oversight of the agency by accepting MI5’s terms. “The tribunal has disabled itself when considering the lawfulness of this regime. You were told just enough for them to say you had been told,” he said. “As a result, you have failed in your oversight role.” While the Security Service audited itself, the IPT did no such thing, said Jaffey. “The tribunal did not plug any gap in oversight.” Jaffey invited the tribunal judges to let the intelligence agencies amass all the information held on judges personally, and then to assess whether that mass of data was held proportionate to the risk that the judges posed a national threat. “The fact that the material is useful to the agencies does not make it proportionate or necessary,” he said.