There is a fundamental paradox confronting a democracy: is a secret agency needed for its safety as a democracy? If so, is the secret agency once established conscious of, and respectful of, civil liberties – that is, how is the secret agency using its judgement when ensuring the safety of the citizen while, at the same time, legally depriving the citizen of some loss of liberty or rights? And how does a democracy manage a secret agency without losing control of it, living with a part of government that behaves in authorised contradiction to the government’s primary democratic values? I call this the Dark Conundrum.

Somewhere in my political development I came to accept the paradox that democracies need secret agencies empowered to gather information on their citizens for the purpose of giving security to that democracy. I assume that most people have, however reluctantly, however unconsciously, adopted this position. Those taking the libertarian position argue that the traditional police forces should be able to manage terrorism – murder is murder regardless of motive; a civilian-made bomb is a bomb regardless of the purpose for which it was made – and that we do not need a secret agency that can do more damage to people than it gives protection.

I felt for my own intellectual clarity that I had to come to a defensible libertarian reconciliation with the Australian secret security agencies. By the nature of such organisations the reconciliation cannot be comfortable. Nor can the reconciliation be stable, and it is sometimes seriously disputed. The libertarian has to be in permanent critical engagement with the secret security services, using whatever means it can. As citizens we know that the effectiveness of secret organisations is patchy; we know that they are prone to chafe at civil-liberty niceties, and use dangerous justification for their breaches. Finally, there are those complicated and shaky ‘crises’ where secret organisations turn to military force, and we know that military forces are themselves imprecise crude instruments for the settlement of political disputes and that almost inevitably their use generates new crises and inhumane error.

The thing about the military-force solution is the ignored possibility that our side may lose. Then what? It seems in Iraq and Afghanistan, Libya, Egypt and probably Syria, we are unable to achieve the result that in some instances we fought and died for.

So the imperfections I have identified – instability of day-to-day political conditions and the rarity of clear-cut dangers – do not give the citizen a sure footing for bringing violence to bear on violence.

My lifetime of writing and political experience has taught me that secret agencies are prone to severe politicisation, error and private agendas against the public good. As I watch secret agencies in this country and in the wider world, I have learned how treacherous they can be, how dangerous it is to train thousands of agents in the black arts of secret agencies. Combined with the technologies of surveillance and counter-surveillance, and intelligence and counterintelligence, of disinformation and trickery, it is now a more volatile mix than ever before in our history.

One of the ways things may go for us on this planet is that the whole apparatus of espionage and counterintelligence will collapse because every nation state can know what it wishes about another. And that no citizen will ever again feel that what they have traditionally called their ‘private life’ will ever exist.

All this, for the state and for the individual, may not be such a horrific thing.

I’m afraid we have to live with a permanent disquiet; however, there are quite a few things about ASIO that we can fix to dispel some disquiet. For a time.

Secrets, shame, private treaties and unshared knowledge may be the harmful things, and their removal could be the beginning in our lives of a new and more compassionate era, of evolution, at least for the West. It may represent the continuation of the great reforms of parliamentary democracy – which began with the idea of a royal commission back in the 11th century in England – such as the freedom-of-information acts (ASIO is exempt from these); the concept of transparency in government; open diplomacy (the end of secret treaties among nations – we hope); public inquiries; anti-corruption agencies; and ombudsmen. For citizens the changes, such as the opening of our lives to the public through social media, represent an abandon- ing of some forms of privacy, and at the same time have eradicated, to some extent, stigmas and the damaging discriminations based on gender, race, sexual preference, ethnicity and so on.

In the 1970s, ‘coming out’ was a unique and dramatic social action by homosexual men and women in protest against criminalisation and stigma, which used as its tactic the public and private declaration – to friends, workmates and family – of their homosexuality.

Coming out was one of the most remarkable and courageous political acts in human history.

Cicero (born 106 BC) coined the expression salus populi suprema lex esto, which is translated variously. It could be termed a primary law, which outranks all other laws and policies. It has been defined as ‘the health of the people should be the supreme law’ or ‘let the welfare of the people be the supreme law’. Defending his new restrictive national security laws in 2006, the then attorney-general, Philip Ruddock, said ‘the most important liberty is life’. This is, literally, nonsense. ‘Life’ is not usually defined as a ‘liberty’, but is an obvious pre-condition to the construction of ‘rights’ – that is, there have to be living people before there can be rights: those laws and policies negotiated as the living arrangements of a community of people. But he was probably scratching around for the Cicero quotation from his law student days. As a libertarian, I argue that fear of terrorist attack or danger to the lives of the citizen cannot be used as the single overriding arbiter of security and policing legislation in a liberal, humanist democracy. Civil libertarians usually counter Cicero with Juvenal’s (1st century BC) quotation et propter vitam vivendi perdere causas (‘And for the sake of life should we give up all that makes life worth living?’). The most important power that ASIO has is the authority to decide what constitutes a threat. It has to have the support of the government of the day – and, as we saw with Menzies, the prime minister can tell ASIO who to treat as a threat regardless of constitutionality. How do we, as citizens, gauge the level of threat if we have only the security agency (for which finding threats is the justification for its existence) and the government (for which finding threats can be a way of silencing critics) to tell us? And of course, once a restriction on the citizens’ right occurs in an emergency, could it remain after the emergency has passed? (Actually, with a casual glance at our history, I see no evidence of this after World War II – restrictions on liberty and other matters were lifted very soon after the end of the war.) The National Threat Assessment Centre (NTAC), made up of representatives of nine organisations and chaired by ASIO, is the body that analyses threats to the nation. In liberal, humanist democracies the threat to us by terrorism has to be resolved, case by case, by the judgement of ASIO and then with the clumsy ways of democracy, with its oversight structure, whistleblowing, media and scholarly investigations, and by the emergence of socially conscious hacking, think-tanks, well-informed scholars and journalists, statements coming from defectors and from the entity that is seen as threatening. All this can also help us form a judgement outside that arrived at by ASIO. Of course, the ultimate way is the referendum. The Swiss have frequent plebiscites on issues. For us the referendum has, in part, been replaced by scientific polling of public opinion, although the advantage of plebiscites over polling is that they concentrate the citizen’s attention and activate discussion of the issue. The character of the al-Qaeda and Taliban terrorist movements and their related Islamic movements is theocratic, anti-democratic, misogynistic, anti–liberal education, anti-pluralist, inhumane and violently aggressive against those who live by Western values. As I write, there is the possibility that these fanatical religious groups (such as ISIS) may gain control of a nation state (Iraq? Syria? Afghanistan? Nigeria? Pakistan?). Organising and controlling a nation state is a complicated task requiring a different range of expertise than freelance, extra-territorial terrorist activities where the fighters live from pillage and in temporary camps. As well, fanatical religious groups have an inbuilt tendency to fragment, which has happened in Syria. The capture by ISIS of Pakistan – a faint possibility – could increase the magnitude of the threat to the West, including nuclear or chemical weapons or terrorist-controlled-drone scenarios. Of course, our own democracies sometimes run close to being controlled by people who have the fantasy that their beliefs are authorised by a supernatural being. When it comes to morality and the enforcing of that morality, the extreme Christian right shares some of the cruel and anti-democratic impulses and doctrines we see in fundamentalist Islam. If I lost my hands and my sight in a terrorist attack in Australia I would be likely to change the compass of my position away from liberty towards safety. But legal wisdom has taught us that we do not allow victims to decide the guilt or choose the punishment of those who have transgressed against them, and nor do we always factor in the suffering and grief of injured citizens in legislating our liberties. Extract from Australia Under Surveillance by Frank Moorhouse, now available through Vintage.