Dear Nigel

"Nothing to hide; nothing to fear," has been a familiar refrain from securocrats and politicians since at least 9/11 and only bettered for bluster by the challenge: "You're either with us or against us." Both are as anti-democratic as they are wrong-headed in exhorting no privacy for the people or scrutiny for the authorities in open societies that are supposed to be defending, not destroying, their "way of life" under threat from enemies within and without.

And both have been proved graphically wrong by many years of experience of a British public that has learned the hard way that we all have something to protect, if not to "hide", and that personal privacy is not the same as guilty secrecy but essential to dignity, intimacy and trust between people.

Try telling Doreen Lawrence and her family that covert surveillance will never be abused by law enforcers, who will inevitably conflate dissent with disloyalty to justify spying on those who expose state abuse alongside those who threaten public safety and national security.

How dare the police investigate the grieving friends and supporters, instead of those suspected of Stephen's murder. The answer is simple: they dared because they could and the victims had become an irritation, even a "threat" to the reputation of a failing institution.

So one can be shocked but not surprised when the new MI5 director general justifies blanket snooping on entire populations and suggests that even scrutinising and debating his capacity undermines national security. Of course there must be balance between transparency and secrecy if espionage and surveillance are to be effective. But chief spooks in democracies should treasure public trust and be slow to avoid public debate, parliamentary scrutiny and the rule of law.

Yours, Shami

Dear Shami

I don't know anyone in the British intelligence and security community who would argue that ordinary citizens do not have the right to privacy. After all, when they go home, members of this community become ordinary citizens and they and their families want to enjoy the same right to privacy themselves. And the overwhelming majority of them do what they do precisely in order to protect the freedoms that the citizens of the UK enjoy, but which are unfortunately under constant threat from a range of malevolent state and non-state actors.

Within the UK, there are clear laws governing the circumstances in which the intelligence and security community can exercise the powers they have been given. And there are multiple mechanisms in place to ensure proper compliance and accountability. But they are not perfect and there will always be cases when things go wrong.

Much of the current furore about what you refer to as "blanket snooping" is misplaced and based on a failure to comprehend how modern telecommunications work.

In the UK, and in most other western liberal democracies, intelligence and security services are not listening to telephone conversations and reading communications on a massive scale. They don't have the remit or the resources.

What they are doing is passing bulk data through computer programs, looking either for specific key words or for patterns of correlation that can only be discerned by analysing large volumes of data. Individual data items are of no interest or relevance unless they meet some very exact criteria – and for these to be examined, specific permission needs to be sought.

This is not 1984. I have lived in a police state and what we are talking about does not take us close to being one.

Yours, Nigel

Dear Nigel

You are right that we don't live in a police state but complacency is as big a danger to democracy as exaggerating dangers created by accident or design.

I have enormous respect for most police and intelligence professionals but cannot agree that their covert activities are sufficiently regulated by law.

One of the biggest objections to what we now understand as the Prism and Tempora spying programmes is the way in which they were conducted around or outside statute, even while the draft communications bill seeking authority for mass on-line data gathering was dropped for lack of public and political support.

You crystallise the argument against me beautifully. Very bad things can be planned and perpetrated online. There isn't interest or capacity to read all communications, so don't worry about them all being collected and retained for random or targeted viewing later on.

But you misunderstand the nature of privacy. The traditional distinction between patterns and content of communications becomes less meaningful with the web. With an old-fashioned letter or phone call, there's a clear difference between the address on the envelope and what I wrote and the phone number I dialled and what I said.

But when you monitor which websites I visit, you quickly paint a very intimate picture of my life – shopping, banking, politics, health, faith and sexuality.

Bad things happen offline, too. In fact, every domestic dwelling that stands long enough will be a crime scene one day. To say that you're only stockpiling online activity to work out who is suspicious is the equivalent of planting cameras and microphones in every family home lest we become suspects later on. It's a giant leap from a more targeted and proportionate tradition that requires suspicion prior to intrusion over the lives of others.

Yours, Shami

Dear Shami,

I agree that living in a world where so much can be inferred about us from our online behaviour is a cause for concern. But that is the world we have sleepwalked into and it wasn't by design of the intelligence agencies. We have allowed our online behaviour to be commoditised in the interests of convenience and access to cheap communications.

Our online behaviour is analysed and sold on by the service providers on whom we have come to depend to the point where it has now become impossible to know who holds what data on us.

The service providers claim this data is anonymised, but no IT expert I know believes that the measures taken provide real personal anonymity. And anyway, who is responsible for verifying that? And what measures are in place to control the activities of the big IT service providers?

There needs to be a debate about big data. The intelligence dimension is a part – but only a part – of that. And the intelligence dimension is, ironically, the best regulated part and the part most susceptible to being held to account.

Big data changes our relationship with information and requires us to think about privacy in different ways. We need to develop a new set of criteria, new professional competencies and professional standards for handling big data. And we do need limitations on what information can be held by whom and for how long. But as a citizen, I am more concerned by the fact that ill-considered social media postings can haunt people for their entire lives than by the fact that the intelligence agencies hold some data for defined periods that they can only access for legally defined purposes.

Yours, Nigel

Dear Nigel

You take a new tactical, yet all-too-familiar tack. We've been careless for too long and it's too late to reunite genie and bottle. Alternatively, we should have a big data debate and ignore particular injustices and illegalities in the interim. And what's more, you trust security more than business with your privacy.

But there are too many logical and practical flaws in this approach. The "benign state/wicked corporate" argument doesn't work in vast parts of a shrinking interconnected world where international human rights' checks and balances need to be applied without fear or favour.

Don't lecture Russia, let alone China, about good governance and the rule of law while handing blank cheques to your own spooks because they wear white hats and are such thoroughly good chaps.

Nor does the argument work for many consumers who perceive a greater voluntary and transactional relationship with online shopping outlets, which can be more instantly damaged by a compromised data scandal, than with governments on five-year cycles or the secure secret state that enjoys covert and mandatory powers and which lasts relatively untouched forever.

Finally, the public/private divide is increasingly unreal in a world where corporations sell mercenaries, prisons and data to governments in return for global power, influence and wealth. The online is far from the exception to this analysis. It is instead the paradigm. What will guide and protect us are ethics and laws based on human rights values applied with an even hand. So if unchecked snooping is wrong in Burma, it's wrong in Britain too. If data-mining without legal authority or explicit consent is wrong for search engines and social networks, it's wrong for the governments of great democracies as well. The dystopian future of 100 movies beckons and we resist the call.

Yours, Shami

Dear Shami

I entirely agree with you that "unchecked snooping" is unacceptable. Intrusive surveillance should always be a last report and should be used with discrimination and self-awareness.

If people actually read some of the documents stolen by Edward Snowden and published in the Guardian, rather than simply relying on misleading headlines, they would see that the intelligence agencies go to extraordinary lengths to ensure that they don't abuse the powers they have been given.

Over three years, NSA erroneously accessed some 60,000 emails of US nationals. As a percentage of global email traffic during that period, that represents some 0.00000000062%. And NSA acknowledged their mistakes rather than, as they easily could have done, sweeping them under the carpet

I yield to no one in my belief in the importance of ethical behaviour, human rights and the rule of law.

I've been to too many places where such values are observed only in the breach to have any other view. And while I would like to see such values achieve universal applicability, I am acutely aware of the challenges to achieving this.

There are many powerful actors who actively oppose these values and we need to be able to defend our own societies against the threats these actors pose.

Liberal democracy is not the natural order: it needs to be fought for and can never be taken for granted.

In an ideal world, intelligence agencies wouldn't exist at all. But our world is far from ideal. And while things remain as they are I believe we need properly empowered and enabled intelligence agencies operating with proper legal authorities and popular consent to defend the freedoms we enjoy.

I happen to think we are much closer to that position than you evidently do. But the great thing about a democracy is that we can have this debate – and try to fix things if they turn out to be unsatisfactory.

Yours, Nigel