The Australian Attorney-General's Department has pushed back at industry and privacy advocate concerns over mandatory data-retention legislation, stating that the leaks on the US National Security Agency's (NSA) surveillance operations by whistleblower Edward Snowden have hastened the need for the regime.

Under legislation currently before the parliament, Australian telecommunications companies would be required to retain an as-yet-undefined set of customer data for two years, not limited to but including call records, address information, email addresses, and assigned IP addresses.

The legislation is being backed up by Australian law-enforcement agencies, which claim that access to the data without a warrant is vital to almost every criminal investigation. Telecommunications companies and privacy advocates, however, warn that the scheme would be a major intrusion on the lives of every Australian, and that the costs of running the scheme will lead to higher prices for internet and phone services.

Special Feature IT Security in the Snowden Era The Edward Snowden revelations have rocked governments, global businesses, and the technology world. When we look back a decade from now, we expect this to be the biggest story of 2013. Here is our perspective on the still-unfolding implications along with IT security and risk management best practices. Read More

Telcos have suggested that existing preservation notices, which agencies can send to carriers, to retain the data for a specific individual under investigation would be much more appropriate than a wide-ranging mandatory data-retention regime.

The Attorney-General's Department, however, claims in its submission to the parliamentary committee investigating the legislation that there are "no practical alternatives" to a legislated mandatory data-retention regime.

"International counterparts have considered the expansion of existing 'quick freeze' preservation notices to cover non-content data as an alternative to data retention. Unfortunately, service providers cannot preserve information that no longer exists. Thus, a preservation notice scheme cannot assist where record-keeping practices are inadequate," the department's acting first assistant secretary Anna Harmer said.

"The purpose of data retention is to introduce a consistent industry standard to ensure that certain limited types of telecommunications data are consistently available."

The Attorney-General's Department also claimed that it needs access to telecommunications customer data because since the leaks of Edward Snowden about the US government's own surveillance regime, it has been much more difficult for Australia's top spy agency, the Australian Security Intelligence Organisation (ASIO), to access content in the way it had in the past.

"Telecommunications data is becoming increasingly important to Australia's law-enforcement and national security agencies as they lose reliable access to the content of communications. This threat has increased significantly since the Snowden disclosures. As such, even where agencies cannot obtain the content of the communications, they have historically often been able to use metadata to determine how and with whom a person has been communicating," Harmer said.

"The ability of agencies to map networks through metadata is an important investigative tool."

While it is not clear exactly why the department believes Snowden has impacted the government's ability to intercept the content of communications, several technology companies including Apple, Google, and Facebook have moved to encrypt their communications on the back of the Snowden revelations.

"Our friends at Apple and Google have probably not helped the situation hugely by introducing operating systems that forensics unfortunately cannot look at," Australian Federal Police (AFP) assistant commissioner and national manager for high-tech crime operations Tim Morris said in November.

On the same day that it was revealed that Immigration and Border Protection referred journalists who had contacted whistleblowers in offshore detention centres to the AFP for investigation, the Attorney-General's Department claimed that "legitimate whistleblowers" would be protected from prosecution using retained data.

"To the extent that concerns relate to the disclosure of the identity of legitimate whistleblowers, it is important to note that such persons have specific protection under the Public Interest Disclosures Act 2013 (PID Act). The effect of those protections is that disclosures by legitimate whistleblowers are not criminal acts. Accordingly, telecommunications data would not be available by reason of the disclosure," Harmer stated.

The department has claimed that narrowing the agencies that can access the data to those investigating criminal activity, such as the AFP, as well as oversight by the Commonwealth Ombudsman, reduces the impact on the privacy of all Australians.

However, agencies that have had their access to customer data taken away can apply to the attorney-general to be added back in after the legislation has come into force.

The Australian Securities and Investments Commission (ASIC) has not made the shortlist of agencies to be included in the scheme, but in its own submission said it should be included.

"In light of ASIC's explicit, extensive, and long-standing criminal law-enforcement functions, there does not appear to be logical reason for its exclusion from the primary definition in the Bill of a 'criminal law-enforcement agency'. In particular, ASIC is not aware of any specific submission or suggestion to the effect that it has either misused its existing powers under the [Telecommunications (Interception and Access) Act] or should have them removed," the agency said.

In an attached report to the Attorney-General's Department's submission, the Australian Government Solicitor found that there would be significant private information retained under the scheme. For example, if a telecommunications customer calls a gay support group, that call record alone would identify information about a person's sexual orientation.

Despite this, the solicitor found that the "narrowing effect" of the legislation in limiting access to the data would enhance privacy.

Parliament is set to resume in February, and Attorney-General George Brandis has said that the data-retention legislation requires urgent passage through the parliament.