Intelligence doesn’t have to be, and rarely is, James Bond-esque. Indeed, 007 is the world’s worst spy: everyone knows who he is, meaning he is compromised, exposed and susceptible to blackmail before he even arrives to save the day.

But the September 11, 2001, terror attacks, and the all-encompassing “war on terror” that followed, did have a major transformative impact on the handling of secrecy and surveillance activities in government programs.

One impact has been the growth of executive authority in national security. This has exacerbated demonstrable tensions between civil liberties, trust and secrecy. As noted by former ASIO chief David Irvine, the public will always have some degree of suspicion regarding the secret function of intelligence agencies.

Read more: Explainer: how the Australian intelligence community works

There is much to be gained by having an electorate better educated in the work of the Australian Intelligence Community (AIC). An informed public is better placed to question or accept the need for Australia’s intelligence agencies. This extends to the merits of expanding budgets, powers, oversight and responsibilities.

A war of choice

In the botched hunt for Iraq’s alleged weapons of mass destruction (WMD) stockpiles in 2003, the “slam dunk” assessments used to build a preemptive war rationale corroded public confidence in political leadership and the intelligence industry.

The WMD fiasco was accompanied by an inexhaustible antagonism around the world about the role of spies and intelligence agencies.

It is now widely accepted that Iraq has been a strategic disaster. Yet many core public debate points remain conflicted and uncertain. To what extent should an intelligence “failure” be traced back to failings in domestic leadership, a selective approach to intelligence estimates and a culture of political spin that seeks external scapegoats?

Cynicism about the credibility of the intelligence sector has not abated. US President Donald Trump, a conspiracy theorist who famously questioned former President Barack Obama’s citizenship, has maintained his theories into the Oval Office. Trump has both embraced and distorted the term “Deep State”. This emotive term is based on the idea that security elites and institutions are deliberately attempting to undermine (or potentially overthrow) the government. Trump has even claimed that America’s spies are acting like Nazis.

His imagery of a corrupt, rogue and monolithic intelligence bogeyman out to topple him is downright bogus. But its rhetorical simplicity plays into the public’s pre-existing fears about a shadowy political world.

Read more: Why does a president demand loyalty from people who work for him?

At the same time, push-back against Trump’s political tactics and self-serving showmanship doesn’t mean the intelligence community should be immune from criticism and accountability.

The CIA, for instance, has a lengthy history of pushing or breaking moral and legal boundaries in locations such as Latin America. Ditto the Iran-Contra affair.

US intelligence has also been accused of monitoring human rights workers and harassing civil society groups and social dissenters engaged in legitimate political activities.

Likewise, in Australia in 1977, the Hope Royal Commission on Intelligence and Security unearthed deep-seated problems with principles of propriety, including legality, within ASIO.

The end result of inquiries into the actions of the intelligence community often raises more questions than answers, which feeds public mistrust and anxiety about intelligence and the function of government.

As we have matured, successive Australian governments have extended and deepened the responsibilities of the intelligence community. It is imperative that future boundaries set for our intelligence agencies are not crossed. For example, we should ensure that the AIC collects information as needed, not just because it can.

Unnecessary snooping is just one area that speaks to the broader importance of oversight, accountability, and an educated and understanding electorate.

Sunlight as disinfectant

The overall mandate of the AIC is to provide the government with “information” to better enable it to understand the issues confronting it. This, in turn, is meant to facilitate coherent policymaking.

However, good intelligence will not automatically guarantee good policy. Intelligence, after all, is an imperfect science and just one of the tools available to policymakers.

The task of intelligence agencies is not an easy one. Indeed, large budgets, hardworking staff and all the expertise in the world cannot ensure that a threat, or opportunity, will be recognised or acted upon in a timely fashion. Connecting the dots will rarely result in an end-product of absolute certainty.

At the same time, the demands of policymakers should not distort or corrupt intelligence – intelligence must inform policy (not visa versa). Intelligence agencies must “speak truth to power”. Integrity and candour, rather than ideological or personal loyalty, remain core prerequisites in dealing with political leaders.

Yet, the appearance of accountability does not necessarily de-politicise national security. Nor does it prevent overt political pressure, or the manipulation of intelligence operations to justify preordained political conclusions.

Read more: The new Department of Home Affairs is unnecessary and seems to be more about politics than reform

In June 2017, Malcolm Turnbull’s government rolled out the largest and most significant changes to the AIC since its creation.

Checks and balances on government error and excess should remain a vital trademark of a healthy democracy. In this sense, a number of the proposed reforms should be applauded. This includes boosting the role and resources of both the Parliamentary Joint Committee on Intelligence and Security, as well as Office of the Inspector-General of Intelligence and Security.

Read more: Intelligence review must tackle anxiety around information-gathering, privacy and security

These changes also attempt to resolve questions about whether the boundaries and operational principles between agencies can be more clearly drawn. There are hopes the establishment of a new Office of National Intelligence (in place of the Office of National Assessments) will assist in greater co-ordination and complementary cross-agency efforts.

Of course, much of the devil remains in the detail. And a number of oversight gaps remain, including how to best protect whistle-blowers who expose unethical or illegal behaviour. We are also yet to see the implementation, execution and performance outcomes stemming from this major reform exercise. Further, the simultaneous announcement of new Home Affairs arrangements will demand equal critical scrutiny.

It’s been mooted that intelligence successes are often overlooked, while intelligence failures are widely broadcast. Popular misconceptions, polarisation and arguably excessive secrecy arrangements mean that the AIC will continue to operate in, and need to be responsive to, a backdrop of public misgivings, political point-scoring and conspiracy theories.