Much like in the movie Casablanca, there is much huffing and shaking of heads about activities of the nation’s electronic spy agency, the Australian Signals Directorate (ASD), after allegations emerged that the agency tapped the mobile phones of Indonesian president Susilo Bambang Yudhoyono, his wife and his senior colleagues. “Round up the usual suspects” echoes in my mind.

Indonesian foreign minister Marty Natalegawa has fulminated about Australia’s alleged spying activities. No doubt the political reverberations will continue for some time, but it is important to have a broader perspective on the significance of the ASD.

An agency is born

In the second world war, decrypted military and diplomatic cables made a significant difference to military strategy and, according to some estimates, shortened the war by up to two years.

The Australian signals intelligence organisation of the day, known as the Central Bureau – as well as its affiliated naval and military components – laid the groundwork for the post-war organisation the Defence Signals Bureau, which would in time come to be known as the ASD.

The decryption of Soviet communications at the end of second world war pointed to a “nest of spies” operating in Australia passing sensitive information to the Soviet Union. This revelation prompted the establishment of the Australian Security Intelligence Organisation (ASIO) under Ben Chifley in 1949.

Successive governments since the formation of ASIO have seen the value of signals intelligence and communications security. Indeed, the dual function performed by the ASD as captured in its motto:

To reveal their secrets and protect our own.

Basics of the spy business

Some argue that in an open, liberal-democratic society such as Australia, the more that is revealed about the functions of this organisation the better. But a couple of caveats should be considered.

First, in the spy business, the secret of success is in keeping your success secret. The paradox is that the more openness there is, the more the other side knows about your capabilities and intentions, the more difficult and costly it is to undertake ongoing investigative work, and the less effective are your own collection efforts.

There is also the notion that the public interest is served by these revelations – even if the national interest may not be. This should be taken with some caution. The Snowden revelations undoubtedly have hurt the Five Eyes intelligence partners (Australia, the US, Britain, Canada and New Zealand) far more than their targets.

British intelligence adviser Sir David Omand speculated that Snowden’s revelations to the Russians and Chinese about the extent of US spy activities, may eclipse those made by theCambridge Five (a group of spies recruited by the Soviet Union during the second world war) in terms of damage. Some, such as ex-CIA boss Michael Hayden, are arguing that Snowden’s revelations expose not just some sensitive products, but “the plumbing” of the system.

It is impossible to fathom the cost of replacing, rebuilding or developing new ways of regaining access lost by Snowden’s revelations. Similarly, it is impossible to know how many terrorist plots now may slip through the surveillance net. Some may dismiss such claims as alarmist, citing past issues of waste, misuse, and lack of authority in Australian intelligence that were prevalent in the 1960s.

Reform and oversight

Since the 1960s, the Australian Intelligence Community has evolved. Three royal commissions conducted under Justice Robert Hope in the mid-1970s, late 1970s and mid-1980s resulted in significant and substantive reform, with accountability mechanisms established as a result of these reviews unmatched in the western world.

Reviews by Philip Flood in 2004 and Robert Cornall and Rufus Black in 2011 have also made further recommendations for reform, while essentially endorsing the extant mechanisms as robust and accountable. The accountability mechanism arising from these reviews are reinforced by the enduring powers of a royal commissioner. This is vested today in the inspector-general of intelligence and security.

Relationship with Indonesia

The ASD’s interest in Indonesia has been long-understood and known by the Indonesians. They have also benefited from this through the extensive intelligence exchange in the years since the Bali bombings, which have helped bolster security and stability for Indonesians as well as Australians.

Certainly, the revelations of phone tapping are embarrassing – and Australia needs to work to smooth over ruffled feathers. But these revelations need to be placed in perspective. Privately, Indonesia may well be quite flattered at Australia’s attention. In the past, they have discreetly been most appreciative of what has been shared with them by Australia’s security and intelligence agencies. They also understand that Australia is interested in Indonesia because it matters so much to Australia.

Perhaps, as my colleague Andrew Carr suggests, Australian prime minister Tony Abbott should turn this crisis into an opportunity for greater openness, collaboration and mutual respect.

At the same time, Yudhoyono and Natalegwa need to weigh up the benefits of what having a peaceful and collaborative southern neighbour brings to Indonesian security, stability and prosperity before offering too much more feigned indignation.

• This article was originally published at The Conversation, part of the Guardian comment network