Jaron Lanier, pioneer turned digital sceptic, explains in an extract from his new book why we must take back control

It might not seem like it at first, but I’m an optimist. I don’t think we have to throw the whole digital world away. But there is one particular hi-tech thing that is toxic even in small quantities.

The issue isn’t only that internet users are crammed into environments that can bring out the worst in us, or that so much power has concentrated into a tiny number of hands that control giant cloud computers. A bigger problem is that we are all carrying around devices that are suitable for mass behaviour modification. For example, with old-fashioned advertising, you could measure whether a product did better after an ad was run, but now companies are measuring whether individuals change their behaviours as they browse, and the feeds for each person are constantly tweaked to get the desired result. In short, your behaviour has been turned into a product – and corporate and political clients are lining up to modify it.

Finally, we can draw a circle around the real danger we face. If we could just get rid of the deleterious business model, then the underlying technology might not be so bad.

Some have compared social media to the tobacco industry, but I will not. The better analogy is paint that contains lead. When it became undeniable that lead was harmful, no one declared that houses should never be painted again. Instead, after pressure and legislation, lead-free paints became the new standard.

I speak as a computer scientist, not as a social scientist or psychologist. I can see that time is running out and doing nothing is not an option. We don’t have as much in the way of rigorous science as would be ideal for understanding our situation, but we have enough results to describe the problem we must solve, just not a lot of time in which to solve it.

Seems like a good moment to coin an acronym, so how about “Behaviours of Users Modified, and Made into an Empire for Rent”? Bummer.

Bummer is a machine, a statistical machine that lives in the computing clouds. Since its influence is statistical, the menace is a little like climate change. You can’t say climate change is responsible for a particular storm, flood, or drought, but you can say it changes the odds that they’ll happen. In the longer term, the most horrible stuff like sea level rise and the need to relocate most people and find new sources of food would be attributable to climate change, but by then the argument would have been lost.

Similarly, I can’t prove that any particular person has been made worse by Bummer, nor can I prove that any particular degradation of our society would not have happened anyway. There’s no certain way to know if it has changed your behaviour, but if you use Bummer platforms, you’ve probably been changed at least a little.

While we can’t know what details in our world would be different without Bummer, we can know about the big picture. Like climate change, it will lead us into hell if we don’t self-correct.

Bummer is a machine with six moving parts. Here are the six components of the machine and a description of each part…

Facebook Twitter Pinterest Illustration by James Melaugh.

A is for Attention acquisition

People often get weird and nasty online. This bizarre phenomenon surprised everyone in the earliest days of networking, and it has had a profound effect on our world. Nastiness also turned out to be like crude oil for the social media companies and other behaviour manipulation empires that quickly came to dominate the internet, because it fuelled negative behavioural feedback.

Why does the nastiness happen? In brief: ordinary people are brought together in a setting in which the main – or often the only – reward that’s available is attention.

With nothing else to seek but attention, people tend to become assholes, because the biggest ones get the most attention. This inherent bias toward assholedom flavours the action of all the other parts of the Bummer machine.

B is for Butting into everyone’s lives

Everyone has been placed under a level of surveillance straight out of a dystopian science fiction novel.

Spying is accomplished mostly through connected personal devices – especially, for now, smartphones – that people keep practically glued to their bodies. Data is gathered about each person’s communications, interests, movements, contact with others, emotional reactions to circumstances, facial expressions, purchases, vital signs: an ever-growing, boundless variety of data.

Algorithms correlate data from each person and between people. The correlations are effectively theories about the nature of each person, and those theories are constantly measured and rated for how predictive they are. Like all well-managed theories, they improve through adaptive feedback.

C is for Cramming content down your throat

Algorithms choose what each person experiences through their devices. This component might be called a feed, a recommendation engine, or personalisation. It means each person sees different things. The immediate motivation is to deliver stimuli for individualised behaviour modification.

Not all personalisation is part of Bummer. When Netflix recommends a movie or eBay recommends something to buy, it isn’t Bummer. It only becomes Bummer in connection with other components. Neither Netflix nor eBay is being paid by third parties to influence your behaviour apart from the immediate business you do with each site.

D is for Directing behaviours in the sneakiest way possible

The above elements are connected to create a measurement and feedback machine that deliberately modifies behaviour. The process runs thus: customised feeds become optimised to “engage” each user, often with emotionally potent cues, leading to addiction. People don’t realise how they are being manipulated. The default purpose of manipulation is to get people more and more glued in, and to get them to spend more and more time in the system. But other purposes for manipulation are also tested.

For instance, if you’re reading on a device, your reading behaviours will be correlated with those of multitudes of other people. If someone who has a reading pattern similar to yours bought something after it was pitched in a particular way, then the odds become higher that you will get the same pitch. You might be targeted before an election with weird posts that have been proven to bring out the inner cynic in people who are similar to you, in order to reduce the chances that you’ll vote.

Bummer platforms have proudly reported on experimenting with making people sad, changing voter turnout, and reinforcing brand loyalty. Indeed, these are some of the best-known examples of research that were revealed in the formative days of Bummer. The digital network approach to behaviour modification flattens all these examples, all these different slices of life, into one slice. From the point of view of the algorithm, emotions, happiness, and brand loyalty are just different, but similar, signals to optimise.

E is for Earning money from letting the worst people secretly screw with everyone else

The mass behaviour modification machine is rented out to make money. The manipulations are not perfect, but they are powerful enough that it becomes suicidal for brands, politicians, and other competitive entities to forgo payments to Bummer enterprises. Universal cognitive blackmail ensues, resulting in a rising global spend on Bummer.

If someone isn’t paying a platform in cash, then they must turn themselves into data-fuel for that platform in order to not be overwhelmed by it. When Facebook emphasised “news” in its feed, the entire world of journalism had to reformulate itself to Bummer standards. To avoid being left out, journalists had to create stories that emphasised clickbait and were detachable from context. They were forced to become Bummer in order to not be annihilated by it.

F is for Fake mobs and faker society

This component is almost always present, even though it typically wasn’t part of the initial design of a Bummer machine. Fake people are present in unknown but vast numbers and establish the ambience. Bots, AIs, agents, fake reviewers, fake friends, fake followers, fake posters, automated catfishers: a menagerie of wraiths.

Invisible social vandalism ensues. Social pressure, which is so influential in human psychology and behaviour, is synthesised.

The more specifically we can draw a line around a problem, the more solvable that problem becomes. Our problem is not the internet, smartphones, smart speakers, or the art of algorithms; the problem is the Bummer machine. And the core of the machine is not a technology, exactly, but a style of business plan that spews out perverse incentives and corrupts people.

It’s not even a widely used business plan. Outside of China, the only tech giants that fully depend on this system are Facebook and Google. The other three of the big five tech companies indulge occasionally, because it is normalised these days, but they don’t depend on it. A few smaller Bummer companies are also influential, like Twitter, though they often struggle.

Which companies are Bummer? A good way to tell is that first-rank Bummer companies are the ones that attract efforts or spending from bad actors, such as Russian state intelligence warfare units. This test reveals that there are pseudo-services that contain only subsets of the components, like Reddit and 4chan, but still play significant roles in the Bummer ecosystem.

Jaron Lanier: ‘The solution is to double down on being human’ Read more

But this argument is not about corporations, it’s about you. Because we can draw a line around the machine, we can draw a line around what to avoid.

The problem with Bummer is not that it includes any particular technology, but that it’s someone else’s power trip. You might choose to be treated by a cognitive behavioural therapist, and benefit from it. Hopefully that therapist will have sworn an oath to uphold professional standards and will earn your trust. If, however, your therapist is beholden to a giant, remote corporation and is being paid to get you to make certain decisions that aren’t necessarily in your own interests, then that would be a Bummer.

The problem isn’t any particular technology, but the use of technology to manipulate people, to concentrate power in a way that is so nuts and creepy that it becomes a threat to the survival of civilisation.

If you want to help make the world sane, you don’t need to give up your smartphone, using computer cloud services, or visiting websites. Bummer is the stuff to avoid. Delete your accounts!

This is an edited extract from Ten Arguments for Deleting Your Social Media Accounts Right Now by Jaron Lanier (Bodley Head, £9.99). To order it for £8.49 go to guardianbookshop.com or call 0330 333 6846