Let’s see if this rings any bells. Back in the early 1990s, just as networked computing was taking off and millions of people logged on for the first time, the US government started getting worried. Although still tiny, ‘cyberspace’ was a nuisance to the law. Untraceable paedophile networks were sharing illegal images of children via the internet. Anonymous hackers were stealing intellectual property. Internet trolling was rife.

And so the US Secret Service upped its monitoring of the online world. In May 1990, it launched Operation Sundevil, a nationwide crackdown on hackers. Meanwhile, law-makers tried to pass legislation to force telecommunications companies to hand over their customers’ details and prevent the spread of powerful cryptography software. In essence, the US government was trying to limit the ability of citizens to stay hidden online.

But a few citizens had other ideas. A small band of Californian libertarians, known as ‘cypherpunks’, began developing tools to keep the net free of state interference. They set up an email list, which doesn’t sound all that auspicious, except that this list ultimately ended up predicting, inventing or refining nearly every technique now employed by computer users to avoid government surveillance. (Julian Assange, the Australian activist who in 2006 founded the whistleblowing website WikiLeaks, became a member in 1995, posting as ‘Proff’.)

It was around this time, too, that a potent tool called PGP (for ‘Pretty Good Privacy’) first emerged. It was created singlehandedly by a US programmer named Phil Zimmerman, who felt alarmed at what seemed to him like a concerted and disproportionate push by the law into citizens’ private space. He decided to make the code of his PGP software freely available to all. Today, it remains the industry standard for text- and file-based encryption on the net.

When it comes to matters of privacy, every government reaction has a citizen counter-reaction. This one was known as the ‘crypto-wars’. And, in the end, the cypherpunks won. By 2001, anonymous retailers were everywhere, an anonymous browser allowing users to use the web without anyone being able to track them was in development, untraceable black markets had sprung up online, and the whistleblowing site Cryptome was becoming a thorn in the side of intelligence agencies. Better still, the US government had dropped its investigation into Zimmermann, and PGP was being used all over the world.

Even so, back in the 1990s, the internet was very obscure: the preserve of hobbyists, academics and specialists. To most of us, the question of internet anonymity was irrelevant. But as more and more of us got online during the 2000s and started sharing (sometimes unwittingly, sometimes not) more about ourselves, the question of digital rights and freedoms started to go mainstream. Surveys and polls started to find that internet privacy was quickly becoming a big deal to the man on the street.

Today, in response to another perceived overreach of governments into our digital space, we appear to be entering a second crypto-war. Ever since the US computer professional Edward Snowden blew his whistle on the US National Security Agency in 2013 and we got our glimpse of the extent of various state surveillance programmes, there has been a boom in the use of (usually free) software to keep internet users anonymous. The anonymous browser ‘Tor’– which allows people to use the net without giving away their internet protocol (IP) address – has leapt in popularity. So has Zimmerman’s PGP encryption. And more people are using the ‘dark net’, an encrypted network of sites that uses the Tor protocol to make tracking close to impossible.

Perhaps more importantly, there are hundreds of people working on ingenious ways to block censorship and keep secrets online: projects that are designed for the mass market rather than the computer specialist. So, there will soon be a new generation of easy-to-use auto-encryption email services, such as ‘Mailpile’ and ‘Dark Mail’. And we can expect to see more distributed social networks: services that function without the kind of centralised server that is vulnerable to state snoopers. Take ‘Twister’, a sort of censorship-free version of Twitter. Every user of the platform runs her own copy of the public record of everything that’s been posted, all hosted on her own computer. Everything could be done anonymously, and with this sort of distributed system, censorship would be close to impossible. No one can shut it down because no one owns it.

In short, the battle-lines have been drawn. Troops and materiel are amassing on both sides. Which means this seems as good a time as any to ask what we want from the internet. How should it balance openness and privacy? What about law and anarchy? And as events unfold, is there anything we can do to decide the outcome?

I recently talked to Miguel Freitas, Twister’s chief developer, in Rio di Janeiro. He explained that he was inspired when the UK Prime Minister, David Cameron, admitted that his government considered shutting down Twitter during the 2011 London riots. ‘I tried searching for peer-to-peer microblogging alternatives, but I couldn’t find any,’ Freitas told me. ‘The internet alone won’t help information flow if all the power is in the hands of Facebook and friends.’ So he spent several unpaid months converting the security protocols that underpin Bitcoin into the basis for a secure social media platform.

Twister is part of a trend toward a decentralisation of the net. Another is called ‘MaidSafe’, which is a UK start-up that aims to redesign the internet infrastructure towards a peer-to-peer communications network, without centralised servers. Its developers are building a network made up of contributing computers, with each one giving up a bit of its unused hard drive. You access the network, and the network accesses the computers. It’s all encrypted and bits of data are stored all across the network, which makes hacking or spying far harder.

Nick Lambert, the chief operating officer for MaidSafe, explained the vision to me. When you open a browser and surf the web, it might feel like a seamless process, but there are all manner of rules and processes that clutter up the system: domain name servers, company servers, routing protocols, security protocols and so on. As Lambert explains, that centralisation results in powerful groups – whether governments, big tech companies, or invisible US-based regulators – exercising control over what happens on the net. That’s bad for security, and bad for privacy.

Why do people such as Freitas and Lambert keep trying to make an end-run around state supervision? Their motivations are typically honourable: they want to make sure that people can remain private online, to keep communication open and free of interference from third parties. The difference between now and the early 1990s is that this impulse is no longer confined to computer geeks. Ordinary people care about internet anonymity and are starting to use tools to get it. Facebook recently launched a version of its network on Tor – proof indeed that it’s becoming mainstream.

But this rise in prominence is leading to some very odd political alliances. The posterboys of internet anonymity, Assange and Snowden, are (broadly speaking) libertarians who mistrust government power; Snowden himself was a supporter of the US Republican presidential hopeful Ron Paul. Yet both Assange and Snowden have worked closely with the British newspaper The Guardian, a Left-wing publication with whom they share practically nothing except worries about government surveillance.

It’s not an exaggeration to say that the laws of mathematics tend toward secrecy

The issue cuts across traditional political boundaries, in other words: everyone has their own reasons for supporting it. The 1990s’ cypherpunks – and there are still plenty out there – believe that anonymity online will lead to a libertarian utopia. The liberal Left fears that state surveillance might encroach on the individual right to privacy. The liberal Right worries about too much power in the hands of state bureaucracy. There are many more who don’t really fit anywhere, but simply believe society is better served if people can stay secret. Spend some time with civil liberties groups, and the cleavage between all these different agendas becomes clear. Yet the biggest faultline is probably the one between the libertarians and the social democrats.

In general, the libertarians want to unilaterally ensure liberty through maths and physics embedded in technology; in this way, they believe that they can chip away at the foundations of state power. The social democrats, on the other hand, would like to achieve liberty through laws and democratic consensus – and hope it will chip away only at bits of the state they don’t think serve progressive ends. However, despite their apparent symmetry, the two factions are not evenly matched. In fact, the basic nature of the question would seem to favour the libertarians.

Because of that powerful combination of public appetite and new technology, the means of staying hidden online will only get easier to use, more widespread and ever more sophisticated. And the cypherpunks have physics on their side: it is easier to encrypt something than to decrypt it. (Encrypting is like cracking an egg; decrypting it without the key is like trying to put it back together again.) It’s not an exaggeration to say that the laws of mathematics tend toward secrecy. Although it might feel unlikely at a time when every click and swipe is being collected by someone somewhere, the direction of travel is toward greater online anonymity. In the years ahead, for those who want it, it will be easier to hide online.

For better or worse, then, it seems that we should expect a good deal of anonymity on the internet of the future. And there is a lot to celebrate in that. Those civil liberties activists you hear banging the drum for internet privacy are not tinfoil-hat-wearing conspiracy theorists. Anonymity of all kinds serves a vital social and individual function. Terrorists using encrypted networks get the pulse going, but there are, I suspect, more genuine freedom-fighters using the dark net than there are terrorists (notwithstanding the unhelpful and idiotic conflation of the two). Syrian democrats really do create secret and untraceable chat rooms to co-ordinate activity. Russian dissidents really do need to circumnavigate state censorship of the net. Gay people in the Middle East really do use anonymous browsers to evade the brutal enforcers of state morality.

Then there are the whistleblower sites. The New Yorker, for example, has a ‘Strongbox’ site on the dark net for whistleblowers to share stories and leaks safely. Anonymity increases the scope for holding the powerful to account, and if nothing else, the internet has given states an awful lot more power. In a world in which every bit of our online behaviour is collected, analysed and sold – a world in which governments have free access even to apparently private areas of the civilian internet – there is the potential for serious misuse and individual censorship.

It isn’t only in the more hostile parts of the world that this matters. In most democratic societies, privacy is seen as a vital part of the sphere of the individual, a precondition for all other kinds of political expression. Well-established democracies use secret ballots to ensure that people can register their political views without fear of reprisal (although, interestingly, in the 19th century, the great liberal thinker John Stuart Mill opposed the secret ballot, preferring that people be forced to stand by their decisions). Anonymity contributes to free expression. In the US, lest we forget, the Federalist Papers were anonymously authored.

As it happens, anonymity is written into the very fabric of the net for this very reason. The pioneers of internet architecture were largely academics who wanted to encourage other programmers to work anonymously to improve the design and structure of the network. They were inspired by the model of the blind peer-review journal: the academic way to ensure that ideas are judged on quality not provenance, so that the best ones rise to the surface.

The freedom to decouple your thoughts from the real you has less grand but equally powerful benefits. It can be difficult to explore sensitive or personal subjects when everyone can see who you are. For people with mental health conditions, having a place where you can speak honestly and openly without fear of judgment is extremely important – and the net is where they go to do it. (Witness the outraged response to ‘the Samaritans Radar’, a recent well-meaning attempt by the UK mental health charity the Samaritans to monitor Twitter for hints of suicidal thoughts.)

Anonymity arguably also results in more interesting content. Sites that have barely any moderation are typically more creative than those that are tightly managed by administrators. Take the infamous image-sharing board 4chan, where most people post under the username ‘Anonymous’ – hence the hacktivism group of the same name. 4chan generates an astonishing amount of inventive, funny and offensive material. Its users vie for popularity and notoriety, but the site prides itself on the anonymity it offers them – pitting itself directly against networking sites such as Facebook and Google+, both of which require users to sign in with their legal names (though of course, many people don’t). Lawless zones such as 4chan might be offensive or nasty – but they can also toughen us up. After all, it’s usually by coming into contact with controversial or extreme ideas that we sharpen our own senses, and develop our own moral compass.

None of which is to deny that anonymity has its problems, or that they might become more serious if the net as a whole were to start operating under the cover of darkness. Studies repeatedly find that people behave worse when they believe they won’t be held accountable for their actions. Cyber psychologists think that we are more likely to be more nasty, and more disinhibited, when protected by the screen. One researcher, the psychologist John Suler of Rider University in New Jersey, writing in the mid-1990s, called this the ‘online disinhibition effect’. He’d been studying early online communities and found that people tend to act more aggressively toward strangers when communicating via a computer. According to a recent survey by the polling company YouGov, 28 per cent of adults in the US admitted to ‘malicious online activity directed at somebody they didn’t know’.

44 per cent of Tor Hidden Services are given up to criminality – ‘dark net markets’ that sell almost anything to anyone, and illegal pornography

Unfettered anonymity, where it leads to bullying and harassment, can also undermine free expression, since it prevents others from taking part in public debate and expressing their own views. Twitter has become a hostile place for many – especially female – writers and commentators who are bombarded with threats from anonymous Twitter users each time they raise their head above the parapet. For non-celebrities, it’s often just as bad, though much less discussed. Many people have left the platform as a result.

That can be distressing and hurtful, of course. But far more seriously, the truth is that along with the journalists, human rights activists and dissidents, groups such as Islamic State and serious criminals will be early adopters of anything that can help them stay hidden. Take the Tor network: it started life as a US military-funded research project, then became an open-source charity aimed at ensuring that civil rights activists and journalists can browse the net safely around the world. It is vital for free expression. And yet, according to researchers at the University of Luxembourg, 44 per cent of Tor Hidden Services (websites that use the same protocols to also stay hidden) are given up to criminality – mainly anonymous ‘dark net markets’ that sell almost anything to anyone, and illegal pornography.

Anders Breivik, the Norwegian terrorist who murdered 77 people in Oslo in 2011 urged others to use the Tor browser in his ‘Manifesto’. Islamic State Twitter accounts share information about how to use Tor. Al-Qaeda has created its own versions of Pretty Good Privacy encryption. But the problem isn’t just that secrecy attracts bad company. On dark net markets, almost every possible drug is available. Some venues sell guns, bomb-making instructions, zero-day exploit hacks – which could create a significant opportunity for terrorist groups. It’s also well-established that serious child pornographers use encryption and anonymous browsers to stay one step ahead of the law, making it more or less impossible to rid the net of images of child abuse (hence the UK National Crime Agency’s admission that they are now unable to arrest everyone who downloads child pornography).

These horror stories aren’t sufficient to justify blanket legal restrictions on the use of anonymous technology, of course. Terrorists also use telephones, cars and cash. Nor is it the fault or responsibility of the people that design and create those tools – it’s down to individuals to take responsibility for how they use these tools. But some technology tilts the balance of power toward those breaking the law – and it is, after all, the state’s primary responsibility to provide public safety and security, since from this all other liberties flow. For all the benefits of online anonymity, it is not an absolute right. There should not be parts of the internet that are entirely beyond the reach of the law enforcement, any more than we should accept similar zones of anarchy offline.

At present, though, there is surprisingly little clarity to be found in law about the right to internet privacy. In the US, the right to anonymous political campaigning was established in a 1995 Supreme Court case and, further in 1999, that ‘people are permitted to interact pseudonymously and anonymously with each other’ with the caveat: ‘so long as those acts are not in violation of the law’. Under Article 8 of the European Convention on Human Rights, every European citizen has ‘the right to respect for his private and family life, his home and his correspondence’. But again, there’s the vital clause: ‘in accordance with law’ and ‘necessary in a democratic society’.

To make matters even murkier, quite what ‘privacy’ means is not entirely clear any more, since notions of privacy changed when we all went online. There are no obligations on us to declare who we are each time we communicate in a public place, or buy a magazine in a shop. I can wear gloves and a balaclava on the street if I choose. But under certain circumstances, decreed vital for the purposes of law enforcement, I will be compelled to remove them. Under stop-and-search powers, the police can compel you to reveal yourself. To vote, to attend school, to travel overseas – in almost every part of life there are requirements to reveal who we really are.

As more people start going dark online, it will also become less effective

So it should be online. We all should have the right to be anonymous online if we wish, just as we can offline. We should be free to use Tor, to use fake accounts and pseudonyms, anonymising operating systems, encryption, and so on. But democratically elected governments should have the capability and legal right to break that anonymity if it is necessary and proportionate (and if there is a good legal system and regulation to ensure that these powers are not misused). As anonymising software becomes more popular, as I’m quite convinced that it will, a new approach for law enforcement will be necessary. Traffic or network-level investigation – the sort of bulk data-collection and pattern-spotting that Snowden revealed – will be increasingly unpopular. As more people start going dark online, it will also become less effective.

And so I predict, in response, more agents, more targeted and limited ‘human intelligence’. In layman’s terms: more James Bonds and fewer Edward Snowdens. More bugs in people’s bedrooms and more infiltration of groups. That will require greater investment in digital policing: new people, new skills, new capabilities, and – because it is more morally hazardous – a new legal settlement. That has to cover how public agencies can access our private information online, what the limits are to online anonymity, and under what circumstances it can be legally broken.

When assessing big-picture trends such as these, it’s not always helpful to think in terms of one mass aggregate score. Nevertheless, I think that the overall influence of online anonymity will remain a positive one for individual and social liberty. Anonymous trolls are a nuisance, but you can’t start outing them without also revealing the human rights activist or undercover journalist. That’s the kicker: anonymity is often valuable and important at different times and in different places, which is reason enough to defend it. Because once it’s gone, it’s usually gone for good. As a general rule of thumb, once governments have powers to monitor citizens, they rarely surrender them. Perhaps anonymity in the future will be even more important than it is now.

Online secrecy is not an absolute right, of course; it is, on occasion, in the interests of general liberty that some people are spied upon. But a prima facie right to act unseen is something worth defending. Oscar Wilde once wrote that if you give a man a mask, he’ll tell you the truth. True, but he’s more likely to be mean and nasty while he’s at it. We just have to live with that.