Better safe than sorry? chombosan/Getty

Would you ride in a car that was prepared to kill you? An “ethical knob” could let the owners of self-driving cars choose their car’s ethical setting. You could set the car to sacrifice you for the survival of others, or even to always sacrifice others to save you.

The dilemma of how self-driving cars should tackle moral decisions is one of the major problems facing manufacturers. When humans drive cars, instinct governs our reaction to danger. When fatal crashes occur, it is usually clear who is responsible.

But if cars are to drive themselves, they cannot rely on instinct, they must rely on code. And when the worst happens will it be the software engineers, the manufacturers or the car owner who is ultimately responsible?


People’s attitudes to the issue are also complicated. A 2015 study found that most people think a driverless car should be utilitarian, taking actions to minimise the amount of overall harm, which might mean sacrificing its own passengers in certain situations. But while people agreed to this in principle, they also said they would never get in a car that was prepared to kill them.

Go “full altruist”

“We wanted to explore what would happen if the control and the responsibility for a car’s actions were given back to the driver,” says Guiseppe Contissa at the University of Bologna in Italy, who along with his colleagues has come up with a solution.

The team have designed a dial that will switch a car’s setting from “full altruist” to “full egoist”, with the middle setting being impartial. They think their ethical knob would work not only for self-driving cars, but for all areas of industry that are becoming increasingly autonomous.

“The knob tells an autonomous car the value that the driver gives to his or her life relative to the lives of others,” says Contissa. “The car would use this information to calculate the actions it will execute, taking in to account the probability that the passengers or other parties suffer harm as a consequence of the car’s decision.”

Me, me, me

But there are issues with the idea. “If people have too much control over the relative risks the car makes, we could have a Tragedy of the Commons type scenario, in which everyone chooses the maximal self-protective mode,” says Edmond Awad of the MIT Media Lab, lead researcher on the Moral Machine project there.

Another concern is that people may be unwilling to take on moral responsibility. If everybody were to choose the impartial option, the ethical knob will not help with the existing dilemma.

“It is too early to decide whether this would be a good solution,” says Awad. But he welcomes a new idea in an otherwise thorny debate.



Journal reference: Artificial Intelligence and Law, DOI: 10.1007/s10506-017-9211-z