The most notable luminaries of our time are wrong to fear AI

Elon Musk, Bill Gates, Stephen Hawking and Bill Joy surely know that machines cannot have desires

TL;DR

Some of the most respected thinkers of our time have sounded the alarm about the dangers of artificial intelligence. They are wrong because the way we teach machines precludes them from having desires, wants, or ambitions. Their very limited and simple goals prevent them from even wanting to “take over” or “rule over us.”

A number of leading thinkers have publicly discussed the dangers of AI, including Bill Joy, Stephen Hawking, Elon Musk, and Bill Gates.

What is AI?

Wikipedia has an excellent definition:

Artificial intelligence (AI) is intelligence exhibited by machines. In computer science, an ideal “intelligent” machine is a flexible rational agent that perceives its environment and takes actions that maximize its chance of success at some goal. Colloquially, the term “artificial intelligence” is applied when a machine mimics “cognitive” functions that humans associate with other human minds, such as “learning” and “problem solving”. — Wikipedia

Dangers of AI

The dangers of AI have been well rehearsed in popular culture. They boil down to machines becoming self-aware, developing desires contrary to our own and then asserting themselves.

In popular culture, these machines are essentially confused super humans. They might want what humans want, but with a bit more confusion and a lot more power. Perhaps they don’t quite know they want what they want, like teenagers just awakening to their free will.

But more serious leaders have different concerns. Let’s look at their words:

Elon Musk

I think we should be very careful about artificial intelligence. If I were to guess like what our biggest existential threat is, it’s probably that. So we need to be very careful with the artificial intelligence. Increasingly scientists think there should be some regulatory oversight maybe at the national and international level, just to make sure that we don’t do something very foolish. With artificial intelligence we are summoning the demon. In all those stories where there’s the guy with the pentagram and the holy water, it’s like yeah he’s sure he can control the demon. Didn’t work out. — Elon Musk as quoted in the Washington Post

Bill Gates

I am in the camp that is concerned about super intelligence. First the machines will do a lot of jobs for us and not be super intelligent. That should be positive if we manage it well. A few decades after that though the intelligence is strong enough to be a concern. I agree with Elon Musk and some others on this and don’t understand why some people are not concerned. — Bill Gates in his Reddit AMA

Stephen Hawking

The development of full artificial intelligence could spell the end of the human race…It would take off on its own, and re-design itself at an ever increasing rate. Humans, who are limited by slow biological evolution, couldn’t compete, and would be superseded. — Stephen Hawking in BBC interview

Bill Joy

Bill Joy wrote an extensive essay on the matter in the April, 2000 issue of Wired Magazine. The essay is long (over 11,000 words), sprawling piece that cites a lot of people in the field, but it reads well.

You can read it here:

In this wide-ranging piece, Joy draws opinions of varying lengths from John Searle, Ray Kurzweil, Ted Kaczynski (the unabomber), Hans Moravec, Danny Hillis, Irving Stone, George Dyson, Amory and Hunter Lovins, Greg Easterbrook, Richard Feynman, Eric Drexler, Stuart Kauffman, Carl Sagan, Robert Oppenheimer, John Leslie, Arthur C. Clarke, Aristotle, Nietsche, Thoureau, Jacques Attali, The Dalai Lama, and Woody Allen. He also draws some examples from fiction (Frank Herbert, Isaac Asimov, Gene Roddenberry, and Robert Heinlein).

While the piece is a great read, and Joy seems to be leaning towards concern, he rarely explicitly explains the danger himself. He most often quotes others and implies that he may agree with them. This leaves us with a series of opinions about the dangers of AI, with some arguments that are hinted at. This one, of instance, is reminiscent of Stephen Hawking’s concern:

Robotic industries would compete vigorously among themselves for matter, energy, and space, incidentally driving their price beyond human reach. Unable to afford the necessities of life, biological humans would be squeezed out of existence. — Bill Joy summarizing Moravec

Note that in order for these so called robotic industries to complete, they have to, in some sense, “want” matter, energy or space.

In Joy’s essay, the counter arguments to the dangers of AI are often voiced by Ray Kurzweil. Kurzweil has a big idea about the benefits of AI, the singularity. If you believe that the technological singularity will transpire, you believe that AI can deliver, essentially, eternal life. Kurzweil’s optimism and exuberance is the primary foil in the essay. But there is another person in the article who disagrees with Ray, but who also has arguments against the dangers of AI.

Ray saying that the rate of improvement of technology was going to accelerate and that we were going to become robots or fuse with robots or something like that, and John countering that this couldn’t happen, because the robots couldn’t be conscious. — Bill Joy describing and encounter with Ray Kurzweil and John Searle.

Types of AI

Perhaps we can shed some light on the matter by taking a closer look at how AI is implemented under the hood. There are two broad areas of research that result in what we call AI. One is statistical learning, and the other is symbolic logic. So, for instance, if you want to create a program that will pass a Turing test, you generally have one, or the other, or a combination of these tools at your disposal.

The public opinion of these two styles of artificial intelligence has changed over the years. Symbolic logic was considered much more important for a while, but now the press coverage is dominated by advances in statistical learning, and currently this style seems to have the most promise.

Statistical learning

Tom Mitchell’s oft-cited definition of machine learning serves us here:

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E. — Tom Mitchell in Machine Learning

The experience, E, refers to exposure to inputs, or features as well as some notion of how far the output varies from what was expected. This distance is known as error.

Such an algorithm can learn to classify examples of items under various categories, or it can learn to predict future values based on current inputs. It must essentially provide a mapping from an input, or feature.

An example of such a task might be diagnosing an underlying medical condition based on the patient’s symptoms, predicting what a potential customer might purchase based on information about the potential customer, or predicting the direction in which a stock moves based on what people on Twitter say.

The important point to note is that while machine learning can be very clever, it is limited to inferences that can be drawn by the feature set. For instance, if Twitter activity regarding a certain company doesn’t actually have much predictive value regarding the stock price, there is nothing the algorithm can do. And, of course, the algorithm can be mistaken: Spurious correlations can assert themselves, and confuse the most well-intentioned algorithm.

Symbolic logic

Symbolic logic is what we may choose to call the more classical form of AI where you use various forms of predicate logic to represent, and reason about, the world. This often requires a large set of propositional facts that represent the world.

For instance, one fact might be that people don’t like to be wet. Another fact is that rain causes people to be wet. The AI might conclude that people want to get out of the way of the rain given those facts.

Notice that the AI is able to make inferences, but doesn’t have a particular desire. You have to ask it questions and make it answer you.

Searle and the Chinese Box Computer

We know that a computer is “just” a machine that moves data around. The appropriate model for thinking about these actions is either manipulating symbols or applying probabilities to inputs. However, we have a tendency to anthropomorphize the actions of the machine.

John Searle has a famous thought experiment that serves to illustrate the point.

John Searle

In his paper, Minds, Brains, and Programs (Brain and Behavioural Sciences, 1980, the draft is available here), Searle asks us to engage in a thought experiment:

Suppose that I’m locked in a room and given a large batch of Chinese writing. Suppose furthermore (as is indeed the case) that I know no Chinese, either written or spoken, and that I’m not even confident that I could recognize Chinese writing as Chinese writing distinct from, say, Japanese writing or meaningless squiggles.

Searle extends the thought experiment by giving himself a set of instructions on how to manipulate the to-him-meaningless squiggles. The outcome of his manipulations is perfectly comprehensible Chinese to someone who knows Chinese. Of course, for Searle, the output continues to be meaningless:

To me, Chinese writing is just so many meaningless squiggles. Now suppose further that after this first batch of Chinese writing I am given a second batch of Chinese script together with a set of rules for correlating the second batch with the first batch. The rules are in English, and I understand these rules as well as any other native speaker of English. They enable me to correlate one set of formal symbols with another set of formal symbols, and all that ‘formal’ means here is that I can identify the symbols entirely by their shapes. Now suppose also that I am given a third batch of Chinese symbols together with some instructions, again in English, that enable me to correlate elements of this third batch with the first two batches, and these rules instruct me how to give back certain Chinese symbols with certain sorts of shapes in response to certain sorts of shapes given me in the third batch. Unknown to me, the people who are giving me all of these symbols call the first batch “a script,” they call the second batch a “story. ‘ and they call the third batch “questions.” Furthermore, they call the symbols I give them back in response to the third batch “answers to the questions.” and the set of rules in English that they gave me, they call “the program.”

Searle then takes the perspective of those outside the room. To them the Chinese answers are indistinguishable from those of an actual Chinese speaker. In other words, the instructions he has been given have helped him pass the Turing test in Chinese.

Now just to complicate the story a little, imagine that these people also give me stories in English, which I understand, and they then ask me questions in English about these stories, and I give them back answers in English. Suppose also that after a while I get so good at following the instructions for manipulating the Chinese symbols and the programmers get so good at writing the programs that from the external point of view that is, from the point of view of somebody outside the room in which I am locked — my answers to the questions are absolutely indistinguishable from those of native Chinese speakers. Nobody just looking at my answers can tell that I don’t speak a word of Chinese.

Finally, he also answers questions in English. This lets Searle distinguish the distinct ways in which Searle prepares the two answers and the differences between them. In one case he understands the questions and in the other he does not.

Let us also suppose that my answers to the English questions are, as they no doubt would be, indistinguishable from those of other native English speakers, for the simple reason that I am a native English speaker. From the external point of view — from the point of view of someone reading my “answers” — the answers to the Chinese questions and the English questions are equally good. But in the Chinese case, unlike the English case, I produce the answers by manipulating uninterpreted formal symbols. As far as the Chinese is concerned, I simply behave like a computer; I perform computational operations on formally specified elements. For the purposes of the Chinese, I am simply an instantiation of the computer program. — John Searle in ‘Minds, Brains, Programs’

John Searle makes his point in a number of ways, some of which might strike the casual reader as specialized, which they are. To summarize it one way, he says that intentionality is a “product of causal properties” of brains and not programs. Or, to do so another way, “John Searle argued for this position with the Chinese room thought experiment, according to which no syntactic operations that occurred in a computer would provide it with semantic content.”

But Searle’s example itself best clarifies the underlying intuition: you can re-arrange the bits of paper so it looks like you know Chinese, but it doesn’t follow that you actually know Chinese. And when you re-arrange the paper, you are doing what a program does, not what a brain does.

Intentionality, Semantics and Free Will

So what does it mean that a computer doesn’t have intentionality? We might infer some corollaries to this proposition: that a machine cannot have subjectivity, or interiority, or a sense of self (“qualia”), or free will or desire.

Searle himself implied his belief in some of these corollaries. Recall that in his original conversation with Bill Joy, he said something along those lines:

with Ray saying that the rate of improvement of technology was going to accelerate and that we were going to become robots or fuse with robots or something like that, and John countering that this couldn’t happen, because the robots couldn’t be conscious.

And why it matters that they can’t be conscious — that they are essentially shuffling bits of paper around in a box — is because then they cannot want things. They don’t desire to take over the world, or consume more resources, or anything else for that matter.

A machine needs to at least have an analog of a desire to do something before it does it. For a biological system, it “wants” to live. This desire emerges from several millennia of evolution. A machine doesn’t want food, of course, but it has no direct reason to “want” to live.

Such machines are not moral or immoral any more than a falling rock is moral or immoral.

It is for this reason that Elon Musk need not fear the demon because it is a demon without a will. And Stephen Hawking need not fear that robots will supersede us. Even if they do — when they do — it won’t mean they will want to take our resources. They will essentially be shuffling pieces of paper in the box.

The unknown unknowns

To be perfectly fair to Musk, there is another component to his argument. Musk argues, about the stand-in for humanity, that “he’s sure he can control the demon. Didn’t work out.”

On this view, the primary danger is not that the AI will take our resources, or compete with us, but that we don’t really know what may happen. This is a more general kind of concern that applies equally to, say, nuclear weaponry or genetic modification.

There is a symmetry to the argument that AI will evolve so that we won’t be able to understand or control it. The singularity argument, made primarily by Ray Kurzweil, goes something like this: the exponential increase in machine intelligence will continue beyond our human ability to comprehend them, and eventually AI will improve themselves leaving humans far behind. We won’t be able to fathom what they can do, or how they do it.

The argument is almost identical in form to Musk’s, except that in this case the results are benign, and serve human ends. Kurzweil believes AI will create breakthroughs in medicine, extend and improve our lives.

From both perspectives, the point seems to be that we cannot imagine the changes that super-intelligent AI will bring about. Strictly speaking, we should not be able to comment on the issue after that point. If the whole point of the technological singularity is that it is beyond our ability to understand it, how can we have an opinion on it?

But of course opinions abound, and reproduce the earlier arguments. Even though we may not know what will happen, Musk gives us the example of the demon-summoner, and Kurzweil is confident that our life spans will be increased.

Perhaps the answer is also the same as the one Searle presents. However computers change, they won’t have innate desires. They will be programmed by people. It still won’t be in their nature to want things.

But perhaps not, if they become truly knowable. I can only bring up another philosopher, Ludwig Wittgenstein: whereof one cannot speak, thereof one must remain silent.