Canada's domestic spy service has been trying to figure out ways of obtaining "bulk data" to better feed the holdings of its secretive analytics centre, newly released records obtained by The Globe and Mail show.

A 2012 memo by the Canadian Security Intelligence Service speaks of an intelligence-agency pivot with profound implications for privacy and security.

Details about the kinds of data being sought by CSIS, and even what exactly it considers bulk data to be, have not been disclosed. But the language used by the spy agency is reminiscent of other so-called bulk-data programs embraced by polarizing U.S. and British intelligence agencies since revealed to have been amassing records relating to the everyday transactions of millions of ordinary people.

Story continues below advertisement

How much data do domestic spy agencies now need – about everybody, not just terrorists – to safeguard national security? The Canadian government's collection practices have never been revealed or debated publicly, even as the closest counterparts of CSIS now openly assert they need bulk data to function. Such data sets can vary greatly in sensitivity – allied spy agencies have said they buy some sets on the open market from so-called big-data brokers. But they have also acquired other sets through clandestine methods or by secretly compelling corporations or other government departments to hand over their data.

The memo from July 19, 2012, released under the Access to Information Act, urged all of CSIS to figure out how to better contribute to holdings of the Operational Data Analysis Centre.

This secretive facility, known as ODAC, was first publicly exposed by a scathing Federal Court ruling released in the fall of 2016.

Titled a Data Management, Data Governance Plan, the CSIS memo from five years ago expressed hopes that the centre's analysts could achieve big things – if they had more data. "ODAC provides vital analytical support to CSIS investigations," it reads at one point. Later it adds, "ODAC collection and analysis can help fill important investigative gaps."

But a pressing matter for CSIS was to "identify critical acquisition issues and develop a framework for obtaining bulk data," the memo said. It also urged intelligence officials to "increase the number of data sets available to ODAC (and eventually the service) to enrich its analytical capability."

Such imperatives are intriguing when you consider what CSIS's closest allies were doing at that time.

By 2012, a bulk telephony metadata program in the United States had existed for years, and filled its intelligence-agency coffers with the phone logs of American citizens. This material was secretly compiled in great volumes, on an ongoing basis, by the U.S. government from phone-company records.

Story continues below advertisement

British spy agencies were doing such collection too – and much more. By amassing what they call bulk personal data sets, British intelligence analysts were mining everyday records held by corporations and other government departments. These records reflect the communications chains, travel histories and financial transactions of millions of people.

It is unclear how much CSIS analysts – who face relatively strict privacy rules – could replicate such programs. The U.S. and British programs raised their own questions about lawfulness after their essence was leaked in 2013, by the former U.S. intelligence contractor Edward Snowden.

Fallout is ongoing in Britain, with a privacy lawsuit forcing British spymasters to come clean about their data dealings. "Without the haystack, one cannot find the needle," a senior deputy director of the MI5 wrote in an anonymized affidavit released last year.

The document explicitly states that British intelligence has amassed enormous bulk-data holdings and needs all of them to thwart terrorist attacks. Data analysis gets more powerful when you have more sets of data. The MI5 affidavit speaks of a time, in 2004, when British spies overlaid several data sets, and how this helped a identify a single suspected suicide bomber from a vague tip that initially would have had intelligence operatives run down 27,000 potential leads.

Federal officials in Canada won't say much about bulk data, but they do point out Canada cannot compel it the same way American and British spy agencies have.

"Our laws do not authorize the government to direct Communications Service Providers (CSPs) to 'do, or not to do, a particular thing' or otherwise require them to provide telecommunications data to Canadian intelligence agencies," wrote Jean-Paul Duval, a spokesman for Public Safety Canada, the federal ministry that oversees CSIS, in an e-mailed response to questions from The Globe.

Story continues below advertisement

When the Federal Court of Canada exposed ODAC last year, it urged CSIS to stay mindful that "strictly necessary" is a term that remains the law. Parliament put this limitation on what records CSIS can collect to prevent "an overly expansive interpretation of the agency's mandate," the court said in a written ruling.

The 14 specially cleared judges who approve CSIS intelligence officers' wiretap warrants complained that no one ever told them about ODAC during its 10 year of operations.

In their written ruling, the judges raised questions about CSIS warehousing data indefinitely. The crux was a finding that CSIS had held onto the associated data of innocent people by never purging records relating to their phone activity and Internet trails, after these people were in contact with people whom CSIS considered targets.

Nearly all government intelligence watchdogs in Canada made cryptic criticisms last year about federal spies dealing in data. One important one occurred last September, when the Security Intelligence Review Committee urged CSIS to put "an immediate halt to its acquisition of bulk data sets."