The past decade or so has seen some spectacular advances in artificial intelligence, but is there a dark side to this brave new world? That was the discussion topic tackled by a panel of computer scientists and ethicists at this year's meeting of the American Association for the Advancement of Science.

There is no denying the impact of the information technology revolution on our economy. From the time that personal computers started infiltrating the workplace, there have been impressive gains in productivity. At the same time, there's been an uncoupling of the traditional link between productivity and employment; unlike in years past, the benefits have not been felt by many—or even most—in society. That was the central message from Moshe Vardi's talk.

A professor of computational engineering at Rice University, Vardi said that technology has been destroying jobs since the industrial revolution—one only needs to look at the role of horses in transportation as an example. But in the past, those jobs have been taken by machines designed to do a specific thing, like weaving cotton. Now, Vardi argued, we're facing the possibility of machines that may be better than humans at nearly everything.

Vardi raised the concerning possibility that an over-reliance on automation and AI could have the same effect on our economy as the Roman dependence on slaves. "Can our economic system deal with labor participation rates below 25 percent? Below 50 percent?" he asked. The solution in ancient Rome, he pointed out, was bread and circuses or life as a legionary.

Getting ahead of that problem now, before the technology becomes entrenched—known as the Collingridge dilemma—is extremely important, according to Yale ethicist Wendell Wallach. Wallach explored the topic of robot morals in his talk, calling for the field of robotics to spend a proportion of its research funds on studying the ethical, legal, and social implications (ELSI) of the technology, in much the same way as the field of genomics. (Since the beginning of the Human Genome Project through today, the National Human Genome Research Institute has had a federal mandate to devote 5 percent of its budget on ELSI research.)

"We need to intervene in concerted ways to shape emerging technology to prevent it from becoming a dangerous master," Wallach argued. Autonomous cars give us a good example—in recent months there has been much discussion of self-driving cars and what's known as the "trolley problem." This is a sort of automotive Kobayashi Maru, where an autonomous car encounters a problem with no good outcomes—crashing into a group of toddlers and saving its occupants or crashing into a tree to save the toddlers, killing the occupants.

Wallach thought that it would be a problem with no obvious ethical answer. "Driving is not a boundedly moral activity," he said. Rather, it's "a social practice." While it's a question that has vexed many, none of the industry experts that we've challenged with it in recent months have had any possible solutions. Most have told us that sensor technology is still a decade or more from discerning whether the object to avoid is a bus packed with children or a tractor-trailer.

Ethicists and social theorists should be embedded in design teams, Wallach told the audience. And researchers and engineers ought to consider who or what is responsible if anything goes wrong with the systems they design.

An important line in the sand needs to be a ban on developing autonomous weapon systems—a call that we reported on last year. "Machines must not make decisions which result in the death of humans," Wallach said, referring specifically to AI or robots programmed to choose their own targets on the battlefield (as opposed to a self-driving car in the trolley problem above).

Machines programmed to kill are mala in se, Wallach argued; machines are unpredictable and cannot be fully controlled, and attribution of responsibility is difficult if not impossible. They would undermine the foundational ethical principle that a human agent (whether individual or corporate) is responsible and accountable for an action, and battlefields of autonomous weapons run the very real risk of removing any last shreds of humanity from the practice of war.

Encouragingly, Wallach believes that the AI community is broadly on board with these ideas, rightly recognizing that society will look on potential Miles Dysons with opprobrium.