Don’t wait for it to go bad Erik Tham/Getty

Artificial intelligence – what’s the worst that can happen? For Roman Yampolskiy, a computer scientist at the University of Louisville in Kentucky, the sky’s the limit. Working with hacktivist and entrepreneur Federico Pistono, he has come up with a set of worst-case scenarios for a potential malevolent AI, from “enslaving mankind” to “destroying the universe”.

Yampolskiy argues that anticipating as many negative outcomes as possible – just as cybersecurity researchers do when looking for vulnerabilities – will help us guard against disaster.

“The standard framework in AI thinking has always been to propose new safety mechanisms,” he says. But looking at the issue with a cybersecurity mindset puts a different spin on things: starting with a list of all the things that could go wrong will make it easier to test any safeguards we may eventually want to put in place.


Some of the catastrophes that Yampolskiy and Pistono envisage involve turning us against ourselves. In one scenario, an AI system unleashes a global propaganda war that sets governments and populations in opposition, feeding “a planetary chaos machine”.

The work was paid for by a fund set up by tech entrepreneur Elon Musk, who has described AI as humanity’s “biggest existential threat”. Stephen Hawking has voiced similar fears.

Unfounded fears?

Not everyone shares their concerns, however. Mark Bishop at Goldsmiths, University of London has argued that such fears are exaggerated. Andrew Ng, the Silicon Valley-based chief scientist for Chinese internet giant Baidu, has compared worrying about the rise of killer robots to worrying about overpopulation on Mars.

But Yampolskiy cites the example of Microsoft’s Twitter chatbot Tay, which recently went rogue when it was tricked into spewing racist comments . Although relatively inconsequential, Yampolskiy says the incident reveals the unpredictability of such systems. “The researchers did not anticipate this outcome,” he says.

“I would like to see a sudden shift to where this is not just a field where we propose solutions,” he adds. “I want this interplay where you propose a safety mechanism but also ask can we break it?”

Noel Sharkey, an AI researcher at the University of Sheffield, UK, agrees that an approach to testing inspired by cybersecurity is a good idea for any system, especially autonomous weapons.

But like Bishop and Ng, he is sceptical about the more general threat of AI. “The idea of a malevolent, superintelligent, general artificial intelligence is still in the realms of science fiction and speculation,” he says.

Reference: arxiv.org/abs/1605.02817