The Obama administration has reached a deal with a number of technology giants, allowing the companies to disclose more information on customer data they are compelled to share with the government.



Announced on Monday, the transparency arrangement ends months of legal wrangling between the companies and US intelligence agencies before a secret surveillance court, to compel the disclosures.



The disclosures are to be nonspecific, listed by the thousand and subject in some cases to a six-month delay – speaking to the large quantities of data that the government still plans on collecting from its technology partners. In order to be more specific about the amount of data turned over, the companies must be less specific about the type of data it is.

The deal also explicitly points to a delay of up to two years on revealing information on data collected under surveillance programs the National Security Agency may yet develop.

But the deal also purports to shed far more light than ever on a question the intelligence agencies have been extremely reluctant to address – the number of people affected by NSA surveillance.

The Justice Department said the transparency deal also applies to phone companies that turn over, on a daily basis, the records of every phone call made in the US. The phone companies have not exhibited the same agitation for transparency in the wake of the Edward Snowden revelations as have tech firms.

“The administration is acting to allow more detailed disclosures about the number of national security orders and requests issued to communications providers, the number of customer accounts targeted under those orders and requests, and the underlying legal authorities,” attorney general Eric Holder and director of national intelligence James Clapper said in a statement on Monday.

The statement said the transparency move was commensurate to President Obama’s recent speech on the issue of government surveillance.

“Through these new reporting methods, communications providers will be permitted to disclose more information than ever before to their customers.”

The new arrangement addresses a major grievance held by Google, Yahoo, Microsoft, Facebook and LinkedIn, which all joined a coalition called Reform Government Surveillance in order to pressure the administration into reassuring their customers about the propriety and legality of giving vast amounts of data to the NSA, FBI and other government agencies. It does not curtail the amount of data demanded, which is another demand of the coalition.

Previously, such companies were barred from announcing how many orders they received from the secret Fisa court. Transparency reports, of the sort published by Google and, more recently, Apple, typically list ordinary criminal or law enforcement data requests, but not anything related to the bulk surveillance programs disclosed by the Guardian, Washington Post and others thanks to Snowden.

In a letter to the companies disclosed by the Fisa court on Monday, deputy attorney general James Cole outlined two paths for additional surveillance transparency.

Every six months, companies can now publish reports listing how many thousand National Security Letters, a form of nonjudicial subpoena from the FBI, they receive, as well as how many customer accounts those letters affect, also listed by the thousand.

Similarly, Cole announced that firms can publish traditional, so-called Fisa orders for content, also by the thousand, along with how many of their customers' “selectors” – a surveillance term indicating a user identifier, like an email address or a screenname – are addressed. Selectors serve as inexact proxies for individual users.

Those same customer selectors and Fisa orders for metadata – such as the “to”, “from” and “subject” line of an email – are now permissible for the tech companies to publish, every six months and in bands of 1,000.

But Cole said that the foreign-focused surveillance orders – orders that have included Americans’ communications “incidentally” collected and subject to usage restrictions – are subject to a six-month delay, meaning tech firms’ periodic transparency reports may publish information on the National Security Letters and Fisa orders they receive, but not covering the same period.

Additionally, once the NSA or other government agency develops a surveillance effort on “a platform, product or service (whether developed or acquired) for which the company has not previously received such an order”, the firms must wait two years before disclosure of its existence. Such “new capability orders” would subsequently be subject to the same biannual reporting requirements after the two-year period expires.

Should firms wish to be more specific as to the number of data requests they receive, the deal Cole unveiled allows them to report customer data requests in batches of 250, but only if they conflate National Security Letters and Fisa orders.

The Justice Department clarified that the companies that provide the phone records of every call made and received within the US are also covered under the deal. But the terms of their disclosure – not publicly sought by the telecoms companies themselves, which have less of a culture of transparency than their tech counterparts – are not yet decided, owing to the uncertain contours of bulk phone records collection that Obama wants to transfer out of government hands.

“The result of that transition will determine the manner in which data about any continued collection of that kind is most appropriately reported,” a footnote in Cole’s letter to the tech firms’ lawyers stated.

Since the Guardian and other news outlets began publishing documents from the NSA indicating its broad data collection, tech companies have protested that gag orders surrounding the US-based data orders have hamstrung them in reassuring their customers. Yahoo in particular sued to disclose information it said would allow it to demonstrate that it did resist a 2007 surveillance order.

But the new transparency deal only covers information that the companies turn over to the NSA, FBI and other US intelligence and law enforcement agencies, subject to court order or administrative subpoena. It does not apply to so-called “upstream” collection, by which the NSA siphons data directly as it transits communications networks, such as its reported collection of information passing between Google and Yahoo data centres – a practice that has drawn denunciation from the companies after the Washington Post disclosed it.

The five firms that were party to the Fisa court transparency suit agreed to drop their case, according to a document released by the court on Monday. Civil libertarians were optimistic about the deal.

“This is a victory for transparency and a critical step toward reining in excessive government surveillance,” said Alexander Abdo of the American Civil Liberties Union, which filed an amicus brief in the case, in a statement.

Abdo urged Congress to compel the NSA to disclose information about its “upstream” collection, which he called “a significant amount of spying that happens without the tech companies’ involvement”.

Representative Rick Larsen, a Washington Democrat who backs increased surveillance disclosures, issued a statement saying Congress must “still act so these reforms have the force of law”.