“Dance like no one is watching,” the American journalist Olivia Nuzzi wrote in 2014. “Email like it may one day be read aloud in a deposition.”

Nuzzi’s advice could have saved Kim Darroch an awful lot of trouble. The UK ambassador to the US has just been forced to resign after describing Donald Trump and his administration as “inept” and “uniquely dysfunctional” in diplomatic memos that were subsequently leaked to the Mail on Sunday.

Darroch’s crime is hardly one of inaccuracy. The only person in the world who seems to disagree with the ambassador’s assessment is Trump himself, who called the civil servant “wacky” and “very stupid” in a dysfunctional series of tweets on Tuesday.

But there is no denying that the memos were, well, not particularly diplomatic. An important skill of an ambassador to the US is to balance two competing requirements: to honestly report to your superiors the fact that the president is inept, while remaining chummy with the inept president.

Darroch is not the first person to face trouble over seemingly private thoughts becoming public. Emails get hacked and published (awkward revelations, Donna Brazile); microphones thought to be turned off are actually broadcasting (better apologise, Gordon Brown); documents get released under freedom of information or data protection laws (be more circumspect, GCHQ); old blogposts are rediscovered in a new light (get deleting, Ben Bradley) … the list seems never-ending.

Gordon Brown faces the media after he was overheard calling Gillian Duffy a ‘bigoted woman’, 2010. Photograph: David Fisher / Rex Features

It is increasingly clear that there are only really two ways to avoid this sort of gaffe. The first is to avoid ever entering the public eye in the first place. But that is harder and harder to do: you may be retweeted disapprovingly by a famous person, have your Facebook post screencapped and shared in a private group, or just be filmed doing something odd but harmless in public. And that is before we admit that “public” is a sliding scale. A WhatsApp message sent to a group of four that gets reposted to a group of 20 is hardly “public”, but if you are the schoolchild betrayed by a friend, it can nonetheless be life-ruining .

And so the second is to ensure you have only ever recorded (publicly or privately) a consistent, polite and focus-group-approved opinion on any issue. If you must be blunt, try to ensure it is in a face-to-face meeting, preferably without notes and in one of those tinfoil-lined tents that President Obama used to take with him on foreign visits. Even then, you run the risk of an embarrassing memoir from your interlocutor. So maybe it is best to just keep quiet.

But the death of frank speaking is, speaking frankly, a disaster. A state, a business, even a household, cannot operate without private conversations. The carefully measured words you choose to describe a problem co-worker in an all-staff email are not the same as the harsh language you save for the one-on-one with your line manager. The perfectly fair complaints you have about your flaky friend who always bails at the last minute would be unspeakably cruel to deliver face to face. And the exasperated planning required to deal with an inept president cannot occur if you are forced to pretend, even behind closed doors, that you are dealing with a very stable genius.

Bizarrely enough, Tony Blair was one of the first to raise the alarm. In 2010, just a few months after the Labour party had been ousted from power, he published his memoir, A Journey, and pondered over his mistakes in office. His choices were unexpected. The Iraq war was defended bitterly, even as he wrote he was “now beyond the mere expression of compassion”.

Instead, his regret was preserved for something that many, particularly journalists, hold to be one of his greatest achievements in office: the Freedom of Information Act. “It’s not practical for government,” he told the Guardian in an interview around the release of his book. “If you are trying to take a difficult decision and you’re weighing up the pros and cons, you have frank conversations. Everybody knows this in their walk of life. Whether you are in business – or running a newspaper – there are conversations you want to have preliminary to taking a decision that are frank.”

Blair was pilloried for his comments, but a decade on, the substance, if not the specifics, holds water. The Freedom of Information Act, after all, contains 24 exceptions; an email hacker, or simply an intemperate forwarder, has no such filter. And so frankness is muted, and caution holds sway.

Keeping it private … Barack Obama is briefed in a tinfoil-lined tent on the situation in Libya, 2011. Photograph: Pete Souza/Official White House Photo

Things have not got better since 2010. Pause a moment, and take stock of how much of your life is recorded, archived and searchable that would have been lost to the waters of time just a few years ago. You probably make fewer phone calls, and message more through WhatsApp, Facebook and Twitter. You almost certainly now have a record of your every movement, either tucked away on your iPhone or stored on Google’s servers. And sure, if you were an early adopter you might have posted and tagged personal photos on Facebook a decade ago, but they certainly weren’t annotated by today’s machine-learning algorithm, which can recognise whether you are in a picture by your face alone.

We don’t often think about this vast hoard of data as a privacy problem, per se. No one else can see that message log. Your emails are protected by your company’s head of information security (who now reports to the CEO, not the head of IT). Even Google protects its cloud with ferocity, albeit with the same paternalism that a farmer guards their swine.

That is because we lack the words. There is a form of privacy being violated by these stores. Entrepreneur and activist Maciej Cegłowski calls it “ambient privacy”: “the understanding that there is value in having our everyday interactions with one another remain outside the reach of monitoring, and that the small details of our daily lives should pass by unremembered. What we do at home, work, church, school or in our leisure time does not belong in a permanent record. Not every conversation needs to be a deposition.”

The death of frankness is one symptom of the wider dearth of ambient privacy in our daily lives. The cheery Silicon Valley surveillance state that prevents us from speaking plainly in private is the same one that requires us to warn our children – or be warned by our parents – not to post things online that might harm employment prospects, and the same one that ensures we vet every utterance to make sure that it doesn’t take on some darker meaning outside of the context in which it was presented.

There is no external agent mandating this self-censorship, at least not in the west: in China and Saudi Arabia, Cegłowski notes, the same capacity is used as a tool of social control. In the UK and the US, “We’re using it to show ads. But the infrastructure of total surveillance is everywhere the same, and everywhere being deployed at scale.”

Another framing makes the difficulty of the problem clear. The problem is not data breaches: it is “data exhaust”, the information that is produced as a byproduct of our digital lives. The vast majority of emails are not useful after they are received (or, frankly, even before then). But we keep them because it is the only way we can keep hold of the one in a million that is. A record of every website you have ever visited is useful when you want to remember that killer quote you landed on a year ago, but it is also a record of every sensitive query you have ever Googled. The purpose of Twitter is not to produce a searchable database of all the times prospective MPs have used racial slurs, but that is a dataset the company now has.

The risks of this exhaust would be bad enough if they simply suppressed frankness, muted our freedom to express ourselves and provided a terrifying capability for authoritarian states to co-opt. But the problem is, the playing field isn’t level. There are two groups that float on, unconcerned by the death of ambient privacy.

The first are the angels: those whose innermost thoughts are already safely expressible, who never need to talk behind someone’s back because what they want to say is acceptable to deliver to their face, and who cannot have a throwaway comment come back to bite them because every comment is perfectly thought-out and expressed first time round.

Such people do not exist.

The second group are those who dismiss the very idea of consistency, who elevate rudeness to a virtue and undermine the entire concept of a shared reality. If you build your reputation on consistency and honesty, then a single hypocrisy can be ruining. If you get elected as an MP after you have already been fired from a newspaper for making up – sorry, “sandpapering” – quotes, then, well, you can get away with writing pretty much anything. You could even put it on the side of a bus, couldn’t you, Boris Johnson?

Similarly, it is the height of unambassadorial behaviour to call someone inept behind their back. But if, like Nigel Farage, you make a habit of telling your negotiating partners to their faces that they have “all the charisma of a damp rag and the appearance of a low-grade bank clerk”, how could a leaked letter possibly hurt you? You might even suggest yourself as a successor to the clearly hopelessly biased civil servant. You, of course, are unimpeachable: your reputation for plain speaking is all the proof anyone needs. And for even more plain speaking, your fans can hear you on the radio five days a week!

Kim Darroch shakes hands with Donald Trump, 2018. Photograph: @KimDarroch/Twitter

The threat is not just to politics. The more we adapt to the absence of ambient privacy, the more the world will split: the worst of us rising to the top, their awfulness baked into their reputation and recontextualised as courage or honesty, the rest of us retreating ever further from the public sphere, trying to find new spaces where conversations are unarchived and unpublished.

But in the same way that you can’t fight the climate crisis by simply switching your own habits, you can’t win back ambient privacy on your own. The risks are societal, and they need to be fought on a societal level. You can try, of course. Quitting Facebook is maybe the equivalent of swapping your car for a bike; deleting all your tweets the same as taking the train on holiday rather than flying. These things help, and if everyone did them, they would help an awful lot.

But everyone isn’t doing them. And some of those personal choices may be less than helpful. Swapping plastic bags for canvas ones can take years to pay off; moving from a public Twitter to a private Facebook group gives the illusion of privacy. However, it is still on Facebook, being monitored, archived and algorithmically curated.

“The infrastructure of mass surveillance is too complex, and the tech oligopoly too powerful, to make it meaningful to talk about individual consent,” Cegłowski concludes. “I believe [Google’s Sundar] Pichai and [Facebook’s Mark] Zuckerberg are sincere in their personal commitment to privacy, just as I am sure that the CEOs of Exxon Mobil and Shell don’t want their children to live in a world of runaway global warming. But their core business activities are not compatible with their professed values.

“Our discourse around privacy needs to expand to address foundational questions about the role of automation: to what extent is living in a surveillance-saturated world compatible with pluralism and democracy? What are the consequences of raising a generation of children whose every action feeds into a corporate database? What does it mean to be manipulated from an early age by machine-learning algorithms that adaptively learn to shape our behaviour?”

Individual action is important, though. It can act as a signal of support for the wider fight. It can frighten corporations into shifting their behaviour ever so slightly. And it can help psychologically, by fighting the creeping feeling of uselessness.

Download a privacy app such as Jumbo and use it to lock down your Facebook, sanitise your Google and erase your Twitter. Switch from WhatsApp to Signal, and from Instagram to Snapchat, and everywhere enable the ability to erase your messages after they’ve been read. Turn off “Hey, Siri” and “OK, Google”, and remove Alexa from your house. Fight for frankness.

But if you do think your email might be read aloud in a deposition, maybe follow Nuzzi’s advice. Frankness in the wrong place is a mistake you only get to make once.