Autonomous vehicles will likely become available in the near future. A new article published in Science raises a classical ethical question about how these cars should be programmed: should a car sacrifice its driver if doing so will save the lives of many pedestrians?

The article found that participants generally do want cars to be programmed in this way for other drivers, but they don’t want their own cars to work this way. It’s a potentially lethal form of “Not-In-My-Backyard” for our more automated future.

For this paper, researchers conducted six online surveys between June and November of 2015. Participants were recruited through the Amazon Mechanical Turk platform. Each of the six online surveys included approximately 200 to 450 participants.

Public good vs. individual behavior

In the first survey, 76 percent of participants said that it was morally correct for self-driving cars to sacrifice one passenger to save 10 pedestrians. That’s an overwhelming preference for cars to be programmed in a utilitarian way, reducing the overall number of casualties in an accident. These participants did not express any concerns about programming being too utilitarian.

The second survey showed that participants didn’t think cars should sacrifice their passenger if only one pedestrian would be saved. Their willingness to sacrifice one passenger went up as the number of pedestrian casualties increased. In this survey, participants supported passenger sacrifice even if they were the passenger. This effect was continuous—there was no threshold at which suddenly people felt willing to sacrifice the passenger. Instead, as the number of saved pedestrians increased, so did the willingness to sacrifice the passenger.

However, this result changed when participants were asked to imagine a family member being sacrificed in the car. In this scenario, participants were not only less likely to support passenger sacrifice, the support was significantly lower than support for self-sacrifice.

When the situation was made a bit more concrete, behaviors changed. Respondents to a third survey said they’d be less likely to purchase a self-driving car if it was programmed to sacrifice the passenger to minimize overall casualties. When asked to consider purchasing a car that would sacrifice themselves, respondents reported a 50 percent likelihood of buying one. When asked to consider a car that would sacrifice a family member to reduce overall casualties, this number dropped to 19 percent.

These participants still believed that the programming to reduce overall casualties was the most moral choice—they just didn’t want this kind of car for themselves or their family.

Survey four used a more complex algorithm-based ranking system to assess the same disconnect between moral belief and purchasing behavior. This survey again demonstrated that participants supported the existence of utilitarian, self-sacrificing, self-driving cars for the good of society. However, like the participants from survey three, they did not want to own one of these self-sacrificing cars.

These studies demonstrate a known social phenomenon: people tend to favor global scenarios that will lead to the best societal outcome, but they often don’t want to abide by that decision themselves. Generally when this type of issue arises (the public good versus individual choice), legislation can be used to ensure individual compliance—a classic example of this type of regulation is mandatory immunization for schoolchildren.

An autonomous trolley

The remaining surveys addressed participants’ attitudes about legislation that would mandate utilitarian self-sacrificing programming for autonomous cars. While participants still agreed that cars programmed to reduce overall casualties via self-sacrifice were morally correct, they were reluctant to accept a law mandating this programming. They were also less likely to consider buying one of these cars if regulations compelled this programming.

This paper demonstrates that self-driving cars may be the latest application of a classic ethical dilemma called the "Trolley Problem." It's a way of finding out what people are willing to sacrifice for the overall good, as people are asked how they would control a trolley that has two options: slam into a single person or plow into an entire group.

Research has shown that 80 to 90 percent of people will typically choose to sacrifice one person to save many others. But the self-driving car version of this problem brings this home by forcing people to imagine that the sacrificed person is either a family member or yourself. At this point, the decision-making process begins to change.

As market availability of self-driving cars approaches, ethical issues regarding programming for these machines become increasingly relevant. The authors of this study suggest that there are three major groups that will have influence over programming decisions: manufacturers, consumers, and legislators.

In the absence of legislation, manufacturers will have discretion to make certain ethical decisions regarding the programming of their vehicles, and consumers will be able to influence programming based on their purchasing preferences. If these two forces are allowed to drive market decisions, then it appears that self-sacrificing vehicles may not be widely adopted, since consumers don’t seem to want their own families put at risk for greater societal benefit (even though they agree this type of programming is morally correct). If the demand is low, manufacturers may not make this type of car, which could significantly reduce the overall benefit of self-driving cars.

At the same time, the surveys suggest that these cars may not be widely adopted if this programming is mandated by law. This presents a tricky cost-benefit analysis that legislators and manufacturers will need to consider carefully when planning for these cars to become available for consumers.

Science, 2016. DOI: 10.1126/science.aaf2654 (About DOIs).