The real risk of AI lies in its ability to remove human agency and make us passive consumers, says Paula Boddington, senior research fellow at Cardiff University and author of Towards a Code of Ethics for Artificial Intelligence

Artificial Intelligence is new to humans. Most of the ethical questions it raises are not new, but AI can present us with age-old dilemmas in particularly powerful forms.

The people developing AI are working on exciting, challenging technology at the cutting edge of human endeavour. They generally work in connected, purposeful teams, in an industry which rewards them with money and status; everyone wants to know about their work. It’s hot.

As AI is developed, we must make sure that it genuinely enhances human life and human agency

AI has myriad potential applications, and some are very promising; assistance with medical diagnosis, for instance. But as AI is developed, we must make sure that it genuinely enhances human life and human agency, rather than squeezing our humanity to one side as mere consumers of AI. Some uses of AI may nudge the lives of the masses into something very different from the rewarding, high-status lives of those who actually produce the AI.

Take entertainment. AI could compose and play music, write novels to suit particular individuals and produce computer games that maximally fit our reward and pleasure circuits, tailoring technology to new scientific insights into human behaviour. Could we be permanently entertained, never pick a duff film and always enjoy the music chosen for us? Surely more pleasure is always “good”? Surely it would be better to let an AI finesse and improve human creativity like this?

Not necessarily.

Naturally, we need some pleasure in our lives, but as a species what we long for is a bit of what the AI superstars have – recognition from others, a sense of progress and a sense of meaning. And we don’t always want to have something done ‘better’ by a machine. We want to do it ourselves.

AI could, with savage efficiency, tap into our weakness for repeated hits of pleasure and easily accessible but ephemeral reward

We don’t always want things to be easy, either. Think about how much of the excitement of watching or participating in a marathon is down to knowing how much training has gone into the big day. The exercise of agency is a key part of what it is to be human.

We humans also have a great store of tales of the fate of those who pursued pleasure at all costs. We already see people everywhere, alone in social groups, glued to the fleeting stimuli on their phones. And AI could, with savage efficiency, tap into our weakness for repeated hits of pleasure and easily accessible but ephemeral reward, at the cost of all else we hold dear.

Let’s make sure that our use of AI doesn’t nudge us into a life where we passively chase pleasure after pleasure to the detriment of the full range of experiences and connections and triumphs and failures that make us human.