The Federal Court of Canada has faulted Canada's domestic spy agency for unlawfully retaining data and for not being truthful with judges who authorize its intelligence programs. Separately, the court also revealed that the spy agency no longer needs warrants to collect Canadians' tax records.

All this has been exposed in a rare ruling about the growing scope of Canadian intelligence collection disclosed by the court on Thursday. At issue is how the federal domestic spy service has been pushing past its legal boundaries in the name of collecting data, in hopes of rounding out the holdings of a little-known Canadian intelligence facility dubbed the "operational data analysis centre."

Many corporations and government agencies are now gravitating toward so-called big data computer analytics that can predict patterns of future behaviour based upon records about what has happened in the past. Spy agencies are no different, and the centre in question appears to be the Canadian Security Intelligence Service's equivalent of a crystal ball – a place where intelligence analysts attempt to deduce future threats by examining, and re-examining, volumes of data.

Story continues below advertisement

Read more: CSIS suspends bulk data mining pending new guidelines

Read more: Concerns over Bill C-51 prompt CSIS to brief other agencies on operations

For subscribers: Liberals' delay in changing Bill C-51 raises suspicions

The fundamental question is whether CSIS should be allowed to do this, by means that include retaining information relating to its wiretaps indefinitely, and indiscriminately keeping information about as many people as it can.

What the Federal Court has decided is that such retention violates federal laws. Under the 1984 CSIS Act, the agency's intelligence officers can only hold onto data deemed "strictly necessary" to keep.

Yet, for a decade it appears the spy agency was secretively holding onto phone logs and e-mail trails that it had mapped out around targets of past investigations, even when it was forced to destroy the underlying intercepted conversations. Retaining such communications metadata – or what CSIS calls associated data – would appear to mean the government is keeping records relating to the family, friends and acquaintances of former terrorism suspects.

Such practices are hard to square with Canada's Charter of Rights and Freedoms, and CSIS officials and their Justice Department lawyers suspected Canada's judges might feel this way if ever apprised. The Federal Court, which authorizes CSIS's wiretapping bids, said it had never been given any idea about the spy service's designs on big data. Even when judges finally figured out they should press CSIS on what it was doing with the data associated with wiretaps, it found the spy agency evasive.

Story continues below advertisement

"The Court had never before been fully informed of the existence of the program. The Court, during the hearings, learned that the program had been in existence since 2006 yet it had never heard nor seen any evidence on the matter," reads the ruling.

The ruling from Judge Simon Noël is significant because it is the second time in three years that a senior Federal Court judge has ruled CSIS has not been truthful to the court in its applications for wiretaps. And this time, judges had to team up as a collective – or to meet en banc – on several occasions to call witnesses before determining the spy service was not meeting its "duty of candour."

"It took four days of en banc hearings, several witnesses and five affidavits for the CSIS to explain the 'associated data' retention program and to answer the designated judges' questions," Justice Noël wrote. He stressed that CSIS "cannot abuse its unique position" and "has certainly not enhanced the Court's trust."

But there is no apparent fallout from this for CSIS yet. While the spy agency says it will stop analyzing the contentious data, there are no indications that it will destroy the data.

Following the ruling, Public Safety Minister Ralph Goodale released a statement that was equal parts stern and upbeat.

"I will be pursuing this criticism with the executive management of the service," he said. But Mr. Goodale added that the judges mentioned that CSIS's data-analytics program "has yielded some useful intelligence results." He suggested the program could be bolstered with a few legislative changes. "The CSIS Act is now more than 30 years old and showing its age as global affairs, threat profiles, technology and public expectations have rapidly evolved," Mr. Goodale wrote.

Story continues below advertisement

Though the minister is currently canvassing Canadians on what powers they think CSIS should have, he has never apprised the public on the kinds of frictions CSIS's operational data analysis centre is causing. While CSIS won't speak to what other kinds of data it holds, its officials say the centre's work continues.

That's because there are other ways to populate it. In one telling passage, Justice Noël points out that controversial legislation passed in the Conservative government's dying days has vastly increased CSIS's access to data held by other federal departments. "CSIS no longer needs a warrant to obtain information from the Canada Revenue Agency," the ruling reads at one point.