George Brandis said the terror threat is faced by all in the region. Credit:Alex Ellinghausen Countries needed "common standards as to the nature of the obligation", so that if a US agency were to approach an American company on Australia's behalf, that company would know that agreed legal tests were being met, Senator Brandis said. In mid-2013, less than 3 per cent of counter-terrorism investigations intercepted communications that were encrypted. Today that figure was more than 40 per cent, Senator Brandis said. "Within a short number of years, effectively, 100 per cent of communications are going to use encryption," he said. "This problem is going to degrade if not destroy our capacity to gather and act upon intelligence unless it's addressed." Under existing laws, telco companies have some obligations to help authorities access communications but Senator Brandis said it was "by no means clear" this extended into the rapidly evolving field of decryption.

Therefore the government wants to make sure the law is "sufficiently strong to require companies, if need be, to assist in response to a warrant to assist law enforcement or intelligence to decrypt a communication", he said. The government wants to co-operate with tech and telco firms to stop terrorists "going dark" – the spy term for becoming impenetrable online – and is looking at strengthening legislation to ensure that co-operation. Senator Brandis will ask like-minded countries with whom Australia shares intelligence to make similar changes and also to improve mechanisms for warrants enabling Australian agencies to seek access to data from overseas counterparts and vice versa. Encrypted messaging has become increasingly popular among terrorist groups to plan attacks. Jailed Melbourne Anzac Day plotter Sevdet Besim used messaging app Telegram, as did the Islamic State-linked attackers killed 130 people in Paris in 2015. Australian IS poster boy Neil Prakash was a keen user of the app Surespot during his time as a recruiter and propagandist and recruiter.

There are serious technical and jurisdictional challenges however. With the end-to-end encryption now commonly used by apps such WhatsApp, Facebook Messenger, Signal and Wickr Me, the companies themselves don't have the decrypted messages. The companies are invariably based overseas, often in the United States. Encryption is also vital for the privacy protection of ordinary app and internet users, and many experts warn that any fundamental weakening of encryption software would leave law-abiding users vulnerable to hackers. Given the difficulty of cracking end-to-end encrypted messages during transmission, one option would be to improve warrant-based access to communications at the sender or receiver ends, Senator Brandis said. "At one point or more of that process, access to the encrypted communication is essential for intelligence and law enforcement," he said. "If there are encryption keys then those encryption keys have to be put at the disposal of the authorities."

He stressed Australia wanted to do this in co-operation with other countries. He will put the issue high on the agenda at the next "five eyes" intelligence meeting later this month with the US, Britain, Canada and New Zealand. "I think we've got to take a common position on the extent of the legally imposed obligations on the device-makers and the social media companies to co-operate," Senator Brandis said. Senator Brandis also said wanted to improve the mechanisms for warrant exchanges between Australian agencies and counterparts in the US and other partners. ASIO should for instance be able to provide US counterparts with a warrant signed by the Attorney-General to access data that related to an Australian investigation but "that arrangement does not exist as simply as that at the moment". Countries needed "common standards as to the nature of the obligation", so that if a US agency were to approach an American company on Australia's behalf, that company would know that agreed legal tests were being met, Senator Brandis said.

Mike Burgess, former chief information security officer at Telstra and former deputy director at cyberspy agency the Australian Signals Directorate, said any solution was likely to be imperfect. Terrorists could write their own encryption software or migrate to apps that were based in out-of-reach jurisdictions. But he said it was better to have some coverage of the problem than little or none. "There are really clever evil people and then there are not so clever evil people. We can't just use the debate about the difficulties to say we're not going to do anything," he said. "I personally want to live in a world where reasonable people and companies would say, 'You know what? Under the rule of law, and with the right oversight and a warrant, communications can be listened to when it's needed to protect us.'" Jacinta Carroll, a counter-terrorism expert with the Australian Strategic Policy Institute and former national security official, said traditional communications companies had always helped stop their services being used for crime or terrorism, and laws needed to be updated so that tech and telco firms did the same.

"This means that these companies – whether headquartered in Australia or overseas – must maintain visibility and access to the service they are providing," she said. Former US intelligence chief James Clapper told the National Press Club in Canberra last week the tech industry should use its creativity and innovation to figure out "a way that both the interests of privacy as well as security can be guaranteed".