The new report, by the UK parliament's Intelligence and Security Committee (200-page / 3.57MB PDF) , has highlighted the fact that conflicts between US and UK laws restrict what US-based technology companies can do to help the intelligence services to identify and combat terrorism.

Achieving the right balance between the need to police online content for national security reasons with individuals' right to privacy has long been a challenge for legislators. However, the report is a reminder of the jurisdictional issues that complicate the issue and which put constraints on efforts to force intermediaries such as US technology companies to take a more proactive role in anti-terrorism intelligence gathering.

An ongoing case between Microsoft and the US authorities shows that tensions between the different legal frameworks to do with communications surveillance and privacy in the US and Europe can also serve to frustrate US law enforcement bodies' efforts to investigate serious crimes.

The Intelligence and Security Committee report

The ISC is the parliamentary watchdog that oversees the work of the UK intelligence services. On Tuesday it published a report into the intelligence gathering that preceded the murder of Lee Rigby, a soldier in the British army, in May 2013.

The ISC's report identified a number of failings by the intelligence services but concluded that Rigby's murder, by two extremists, could not have been prevented.

The report revealed that one of the killers, Michael Adebowale, had participated in "a substantial online exchange" with a third extremist in which the killer had "expressed his desire to murder a soldier in the most explicit and emotive manner". The ISC criticised an unnamed communications service provider (CSP), reported to be Facebook, about the approach it had taken to spotting and erasing that content and called for CSPs to do more to flag potential terrorist activity to law enforcement bodies.

"The company on whose systems this exchange took place had not been aware of the exchange prior to the attack," the ISC report said. "However, they had previously closed some of Adebowale’s accounts because their automated system deemed them to be associated with terrorism – yet they neither reviewed those accounts nor passed any information to the authorities."

"We take the view that, when possible links to terrorism trigger accounts to be closed, the company concerned – and other communications service providers – should accept their responsibility to review these accounts immediately and, if such reviews provide evidence of specific intention to commit a terrorist act, they should pass this information to the appropriate authority," it said.

However, the report also identified a number of problems that UK intelligence agencies and law enforcement bodies face in compelling US-based technology companies to monitor for terrorist activity via their services.

The UK's legal framework on surveillance

There are tensions within the legal framework in the UK between rules that require CSPs to disclose personally identifiable data to aid in the prevention and detection of crime and rules that require those companies to respect individuals' privacy.

On the one hand, the E-Commerce Regulations prohibit CSPs from undertaking general monitoring of individuals' communications over their network and the Data Protection Act places conditions on the disclosure of personal data.

On the other hand, rules such as those contained in the Regulation of Investigatory Powers Act (RIPA) mean those same businesses can be ordered to hand over certain metadata relating to communications, not including the content of communications, to law enforcement bodies where certain legal thresholds permitting such disclosure are met. RIPA can also be used as a basis for providing law enforcement agencies with rights to intercept communications.

What are the jurisdictional issues?

Businesses, pretty much regardless of where they are based in the world, face competing demands to protect the personal information of customers and also disclose data that could help law enforcement bodies combat serious crime. Businesses therefore need a solid legal basis to disclose information about their customers.

Whilst it is clear that UK-based CSPs must comply with UK laws, the ISC's report referenced frustrations raised by the Home Office over their attempts to get US-based CSPs to comply with the RIPA regime. The Home Office has conceded that RIPA "contains no lever to compel assistance from overseas CSPs, beyond the power to seek an injunction from a civil court that would require them to do so", and said that that power had "not yet been tested".

US-based CSPs have argued that complying with RIPA could mean they fall foul of US legislation. The ISC said that none of Facebook, Google, BlackBerry, Microsoft, Yahoo, Apple and Twitter had accepted "the UK’s jurisdiction on requests for lawful intercept (i.e. content) for intelligence investigations" and that they would only "provide private information on users under US – and not UK – legal processes".

The ISC said that UK intelligence services can ask partner agencies in the US to submit an application to a US court which would require US-based CSPs to disclose the information to the UK bodies. However, it said that the US courts restrict that practice to "high priority investigations where there is a known threat to life" and that it means that UK intelligence agencies "cannot use this tool in lower priority investigations or in seeking to identify the threat an individual or network may pose to UK national security".

The issue is more clear-cut in emergency situations, according to the ISC's report. US-based CSPs have a legal basis under US law for disclosing customer information to UK intelligence agencies upon their request if they have "a good faith belief that an emergency involving death or serious physical injury to any person requires disclosure without delay".

However, the ISC said that if UK intelligence agencies are merely trying to "establish the risk posed" then the threshold for disclosing that information under US law would not be met. "They cannot therefore use this as an investigative tool as they are unlikely to receive a response from the CSPs," the ISC said.

The ISC's report also looked into another possible way for UK intelligence to obtain customer information from CSPs. The UK has a mutual legal assistance treaty (MLAT) with the US which allows UK and US authorities to share information to help in the investigation, prosecution and combating of crime.

However, the report said the MLAT framework "is not available for use in intelligence investigations where the aim is to determine the threat posed by individuals and there is as yet insufficient evidence for criminal prosecution".

The Home Office has said that the MLAT framework is "too slow" and that it doubts that it will, even with improvements, "ever provide a viable alternative to direct cooperation from communications service providers on interception requests made under RIPA".

However, the ISC said that the UK should try to expand its MLAT agreement with the US to enable sharing of information to aid with intelligence investigations. It said existing initiatives to tackle child abuse images posted online show that there are ways to overcome legal barriers to disclosure .

"The exceptional and long-standing co-operation between the UK and the US on intelligence issues must be utilised to explore an agreed procedure for access to online communications from providers based in the US," the ISC said in its report. "UK citizens are unnecessarily exposed to greater risk while the current situation continues."

Practical considerations

Major CSPs host or transmit a vast amount of content. Some of the US CSPs told the ISC that, in addition to the consideration of service users' privacy, they do not proactively monitor content on their networks because it is not possible to keep track of everything posted. Most act to remove material only when "notified of offensive content … by others".

However, the ISC said that this approach did not help the intelligence and security agencies to uncover terrorist networks or plots and said that CSPs should do more to help, and that regard for user privacy should not stand in the way of efforts to prevent terrorism.

"Several of the companies ascribed their failure to review suspicious content to the volume of material on their systems," the ISC said. "Whilst there may be practical difficulties involved, the companies should accept they have a responsibility to notify the relevant authorities when an automatic trigger indicating terrorism is activated and allow the authorities, whether US or UK, to take the next step. We further note that several of the companies attributed the lack of monitoring to the need to protect their users’ privacy. However, where there is a possibility that a terrorist atrocity is being planned, that argument should not be allowed to prevail."

The Microsoft case

Microsoft has been fighting an order by the US courts to hand over customer data it stored on servers in Ireland to US law enforcement bodies.

Microsoft has argued that US laws that require such disclosure do not apply to data stored outside of the US and said that it would be in breach of EU data protection rules by adhering to the US authorities' request.

That case highlights the conflicts between US and EU legal frameworks in circumstances paradoxical to those referenced in the ISC report.

A greater spotlight has fallen on surveillance practices in the UK and US following the disclosures made by former US National Security Agency (NSA) employee Edward Snowden in the summer of 2013.

The role that technology companies play in assisting intelligence gathering has come in for scrutiny. Many of those companies have taken steps to stiffen security measures, including encrypting data over their networks, in an effort to retain consumer trust on privacy. The ISC's report and the Microsoft case serve as reminders that those companies operate in a challenging legal environment in which the balance between national security and privacy is difficult to achieve.

Luke Scanlon is a technology law expert at Pinsent Masons, the law firm behind Out-Law.com