Service providers caught up in Australia's data retention scheme will have to encrypt customer information, but that's about as much guidance as the Attorney-General's Department offers.

The advice issued by the Department offers scanty information on what constitutes suitable storage infrastructure, and no advice at all on the physical location of retained data.

That's what The Register believes, after accessing two documents, titled “DATA RETENTION Guidelines for Service Providers” and “DATA RETENTION Industry FAQs on the data retention obligations”. Both are dated May 2015 and both bear the Attorney-General's Department's logo and slogan. The Department has twice denied our requests for copies of these documents, but has told us that guidance was issued “In May”.

The FAQs includes a lengthy section titled “Technical Questions” that, at Paragraph 3.2 asks “Will data archived offsite (possibly taking days to access) be compliant under the data retention obligations?” The answer is in the affirmative, so long as offsite storage doesn't unduly delay retrieval of data.

Paragraph 3.3 says “service providers are not required to centralise retained data,” so it looks like service providers won't need to create one cache of all their retained metadata.

Paragraph 3.4 starts with the question “Will service providers have to ensure high-reliability, duplicated data storage?” and offers the following three bullet points in response:

“Service reliability should mirror existing levels. That is, if billing type information is held for extended periods to a level sufficient to satisfy customers and applicable legal obligations, it is expected that information of this kind would continue to be held at that level.”

“For other network information that is currently retained for business purposes, such as for network trouble shooting, that information may be kept to the standard appropriate to achieve that purpose.

The CAC expects data retention systems to function correctly most of the time, but acknowledges occasional, minor disruptions as a result of unforeseen technical issues are a natural incident of the provision of the capability.”

To your correspondent's mind, those points leave the question un-answered. Or perhaps the answer is that retained metadata can reside on the same sort of systems a service provider uses to store their billing information. If that's a single server with a USB drive as backup, tucked away in an unlocked filing cabinet somewhere – not inconceivable at a small ISP – that's where your metadata will reside.

The FAQ also omits any discussion of physical security for retained data. On top of the fuzzy description of storage media, might this mean that filing cabinet could be in a garden shed?

To be fair, there's a review process for the data retention implementation plans that service providers have to submit (and which score the marvellous acronym DRIPs) so the department will have a chance to decline shed-based archiving systems.

Which will mean more costs … but perhaps not capital costs, as the FAQ's paragraph 3.17 tells us that carriers “may outsource technical capability to meet the legislated requirement to keep data”. Which appears to leave open the possibility of a bureau storing data from multiple service providers.

There's good news in the form of an insistence that retained metadata be encrypted: the “Guidelines for Service Providers” says “The data to be retained must be encrypted and protected from unauthorised interference or unauthorised access.”

The Department says “Guidance materials on encryption and the protection of retained data are currently under development and will be made available to service providers in due course.”

That's a sub-optimal situation as service providers have been urged to start work on their DRIPs sooner rather than later. Absent encryption suggestions, it probably won't be any easier to meet DRIP deadlines.

The Register has asked the Department if it has staff on hand capable of developing detailed guidance and assessing DRIPs' suitability as they are lodged. We were told “The Australian Government has established a multi-agency technical team to provide specific technical guidance to the telecommunications industry and to provide advice to the Communications Access Co-ordinator about data retention implementation plans, including security and protection obligations.”

Back in the FAQs, at paragraph 3.8, we get the question “What standards, if any, would service providers be asked to build to, e.g. international standards such as ETSI, or an agency standard?”

The answer is “Service providers are not required to build data retention capability to a particular standard. However, the ETSI retained data standard may be instructive to providers developing new systems.”

ETSI – the European Telecommunications Standards Institute – has two retained data standards.

ETSI TS 102 656, “Requirements of Law Enforcement Agencies for handling Retained Data” (PDF) and ETSI TS 102 657, ”Handover interface for the request and delivery of retained data (PDF) offer very detailed descriptions of how to securely transmit retained data.

The Department tells us that "Further refinements are currently being made to the guidance materials to address questions from industry as they arise and will be made available to industry participants upon request before the end of July 2015." Our request about the publication of guidance on encryption did not receive a response.

We'll therefore assess what we can with the documents dated May 2015 and hope the Department – or a carrier in receipt of any revised documents – helps us out once the revisions arrive.

The Register has considered making the documents we've accessed publicly available: we've decided against doing so as we suspect the Attorney-General's Department may have considered unique steganographic identification before making them available to service providers and that distribution could therefore have unacceptable consequences. If that sounds unduly paranoid, remember that we are dealing with a government that plans warrantless mass surveillance of its citizens, complete with a regime that "protects" journalists by making requests for their metadata the subject of secret warrants! ®