Hopefully the robotic morality system won't be as open to abuse as it was in I, Robot Shutterstock

As we all learned from the 1986 film War Games, machines have the upperhand in warfare when it comes to making logical decisions (such as, the only winning move in nuclear war is not to play). But now it seems the US Navy is not content with that party trick, as it is working on teaching artificial intelligence how to make moral and ethical decisions, too.

A multidisciplinary team at Tufts and Brown Universities, along with Rensselaer Polytechnic Institute, has been funded by the Office of Naval Research to explore the challenges of providing autonomous robots with a sense of right and wrong -- and the consequences of their actions.


Matthias Scheutz, principal investigator on the project, and director of the Human-Robot Interaction lab at Tufts, believes that what we think of as a uniquely human trait could be simpler than most of us thought. "Moral competence can be roughly thought about as the ability to learn, reason with, act upon, and talk about the laws and societal conventions on which humans tend to agree," Scheutz said. "The question is whether machines -- or any other artificial system, for that matter -- can emulate and exercise these abilities." One of the scenarios that could be of particular interest to their Naval benefactors, Scheutz explained, is that of a battlefield.

Say a robotic medic, of which there are sure to be many in our near-future, is ordered to transport a critically wounded soldier to the nearest field hospital. En route, it encounters a marine with a fractured leg. Should the robot stop to administer aid, or continue with its mission? If it does stop to assess the soldier's condition, what if it discovers a life-threatening complication such as a severed internal artery, which requires immediate traction? Applying traction in the field would likely save the marine's life, but it will cause immense pain. Is a robot morally permitted to inflict pain, even if it is for the greater good of saving the soldier's life?

To answer these hypothetical questions, the researchers need to ask yet more questions about the nature of empathy and isolate the essential elements of human moral competence. Once they know what makes people feel, they'll be working on a framework to model human-level moral reasoning in a distinctly non-human computer. "Our lab will develop unique algorithms and computational mechanisms integrated into an existing and proven architecture for autonomous robots," says Scheutz. "The augmented architecture will be flexible enough to allow for a robot's dynamic override of planned actions based on moral reasoning."

Selmer Bringsjord, head of the Cognitive Science Department at the Rensselaer Polytechnic Institute, is working with Scheutz to ensure moral logic is intrinsic to future artificial beings. The lack of an established meaning for what constitutes morality in humans is a challenge Bringsjord will need to tackle. In his team's approach, all decisions a robot makes will face a preliminary ethical check, and if there is a need for a deeper, deliberate moral reasoning then it knows to initiate that process itself.


In order to see this embed, you must give consent to Social Media cookies. Open my cookie preferences.

"We're talking about robots designed to be autonomous; hence the main purpose of building them in the first place is that you don't have to tell them what to do," Bringsjord said. "When an unforeseen situation arises, a capacity for deeper, on-board reasoning must be in place, because no finite rule set created ahead of time by humans can anticipate every possible scenario."

While it may be a long road to teaching robots to feel compassion for humans, it shouldn't be too difficult for the opposite effect, as this video of Japanese astronaut Koichi Wakata saying farewell to his robotic ISS companion is testament to.