Strong AI appears to be the topic of the week. Kevin Drum at Mother Jones thinks AIs will be as smart as humans by 2040. Karl Smith at Forbes and “M.S.” at The Economist seem to roughly concur with Drum on this timeline. Moshe Vardi, the editor-in-chief of the world’s most-read computer science magazine, predicts that “by 2045 machines will be able to do if not any work that humans can do, then a very significant fraction of the work that humans can do.”

But predicting AI is more difficult than many people think.

To explore these difficulties, let’s start with a 2009 bloggingheads.tv conversation between MIRI researcher Eliezer Yudkowsky and MIT computer scientist Scott Aaronson, author of the excellent Quantum Computing Since Democritus. Early in that dialogue, Yudkowsky asked:

It seems pretty obvious to me that at some point in [one to ten decades] we’re going to build an AI smart enough to improve itself, and [it will] “foom” upward in intelligence, and by the time it exhausts available avenues for improvement it will be a “superintelligence” [relative] to us. Do you feel this is obvious?

Aaronson replied:

The idea that we could build computers that are smarter than us… and that those computers could build still smarter computers… until we reach the physical limits of what kind of intelligence is possible… that we could build things that are to us as we are to ants — all of this is compatible with the laws of physics… and I can’t find a reason of principle that it couldn’t eventually come to pass… The main thing we disagree about is the time scale… a few thousand years [before AI] seems more reasonable to me.

Those two estimates — several decades vs. “a few thousand years” — have wildly different policy implications.

If there’s a good chance that AI will replace humans at the steering wheel of history in the next several decades, then we’d better put our gloves on and get to work making sure that this event has a positive rather than negative impact. But if we can be pretty confident that AI is thousands of years away, then we needn’t worry about AI for now, and we should focus on other global priorities. Thus it appears that “When will AI be created?” is a question with high value of information for our species.

Let’s take a moment to review the forecasting work that has been done, and see what conclusions we might draw about when AI will likely be created.

The challenge of forecasting AI

Expert elicitation

Maybe we can ask the experts? Astronomers are pretty good at predicting eclipses, even decades or centuries in advance. Technological development tends to be messier than astronomy, but maybe the experts can still give us a range of years during which we can expect AI to be built? This method is called expert elicitation.

Several people have surveyed experts working in AI or computer science about their AI timelines. Unfortunately, most of these surveys suffer from rather strong sampling bias, and thus aren’t very helpful for our purposes.

Should we expect experts to be good at predicting AI, anyway? As Armstrong & Sotala (2012) point out, decades of research on expert performance suggest that predicting the first creation of AI is precisely the kind of task on which we should expect experts to show poor performance — e.g. because feedback is unavailable and the input stimuli are dynamic rather than static. Muehlhauser & Salamon (2013) add, “If you have a gut feeling about when AI will be created, it is probably wrong.”

That said, the experts surveyed in Michie (1973) — a more representative sample than in other surveys — did pretty well. When asked to estimate a timeline for “[computers] exhibiting intelligence at adult human level,” the most common response was “More than 50 years.” Assuming (as most people do) that AI will not arrive by 2023, these experts will have been correct.

Unfortunately, “more than 50 years” is a broad time frame that includes both “several decades from now” and “thousands of years from now.” So we don’t yet have any evidence that a representative survey of experts can predict AI within a few decades, and we have general reasons to suspect experts may not be capable of doing this kind of forecasting very well — although various aids (e.g. computational models; see below) may help them to improve their performance.

How else might we forecast when AI will be created?

Trend extrapolation

Many have tried to forecast the first creation of AI by extrapolating various trends. Like Kevin Drum, Vinge (1993) based his own predictions about AI on hardware trends (e.g. Moore’s Law). But in a 2003 reprint of his article, Vinge noted the insuﬃciency of this reasoning: even if we acquire hardware suﬃcient for AI, we may not have the software problem solved. As Robin Hanson reminds us, “AI takes software, not just hardware.”

Perhaps instead we could extrapolate trends in software progress? Some people estimate the time until AI by asking what proportion of human abilities today’s software can match, and how quickly machines are “catching up.” Unfortunately, it’s not clear how to divide up the space of “human abilities,” nor how much each ability matters. Moreover, software progress seems to come in fits and starts. With the possible exception of computer chess progress, I’m not aware of any trend in software progress as robust across multiple decades as Moore’s Law is in computing hardware.

On the other hand, Tetlock (2005) points out that, at least in his large longitudinal database of pundit’s predictions about politics, simple trend extrapolation is tough to beat. Consider one example from the field of AI: when David Levy asked 1989 World Computer Chess Championship participants when a chess program would defeat the human World Champion, their estimates tended to be inaccurately pessimistic, despite the fact that computer chess had shown regular and predictable progress for two decades by that time. Those who forecasted this event with naive trend extrapolation (e.g. Kurzweil 1990) got almost precisely the correct answer (1997).

Hence, it may be worth searching for a measure for which (a) progress is predictable enough to extrapolate, and for which (b) a given level of performance on that measure robustly implies the arrival of Strong AI. But to my knowledge, this has not yet been done, and it’s not clear that trend extrapolation can tell us much about AI timelines until such an argument is made, and made well.

Disruptions

Worse, several events could significantly accelerate or decelerate our progress toward AI, and we don’t know which of these events will occur, nor in what order. For example:

Great uncertainty

Given these considerations, I think the most appropriate stance on the question “When will AI be created?” is something like this:

We can’t be confident AI will come in the next 30 years, and we can’t be confident it’ll take more than 100 years, and anyone who is confident of either claim is pretending to know too much.

How confident is “confident”? Let’s say 70%. That is, I think it is unreasonable to be 70% confident that AI is fewer than 30 years away, and I also think it’s unreasonable to be 70% confident that AI is more than 100 years away.

This statement admits my inability to predict AI, but it also constrains my probability distribution over “years of AI creation” quite a lot.

I think the considerations above justify these constraints on my probability distribution, but I haven’t spelled out my reasoning in great detail. That would require more analysis than I can present here. But I hope I’ve at least summarized the basic considerations on this topic, and those with different probability distributions than mine can now build on my work here to try to justify them.

How to reduce our ignorance

But let us not be satisfied with a declaration of ignorance. Admitting our ignorance is an important step, but it is only the first step. Our next step should be to reduce our ignorance if we can, especially for high-value questions that have large strategic implications concerning the fate of our entire species.

How can we improve our long-term forecasting performance? Horowitz & Tetlock (2012), based on their own empirical research and prediction training, offer some advice on the subject:

Explicit quantification : “The best way to become a better-calibrated appraiser of long-term futures is to get in the habit of making quantitative probability estimates that can be objectively scored for accuracy over long stretches of time. Explicit quantification enables explicit accuracy feedback, which enables learning.”

: “The best way to become a better-calibrated appraiser of long-term futures is to get in the habit of making quantitative probability estimates that can be objectively scored for accuracy over long stretches of time. Explicit quantification enables explicit accuracy feedback, which enables learning.” Signposting the future : Thinking through specific scenarios can be useful if those scenarios “come with clear diagnostic signposts that policymakers can use to gauge whether they are moving toward or away from one scenario or another… Falsifiable hypotheses bring high-flying scenario abstractions back to Earth.”

: Thinking through specific scenarios can be useful if those scenarios “come with clear diagnostic signposts that policymakers can use to gauge whether they are moving toward or away from one scenario or another… Falsifiable hypotheses bring high-flying scenario abstractions back to Earth.” Leveraging aggregation: “the average forecast is often more accurate than the vast majority of the individual forecasts that went into computing the average…. [Forecasters] should also get into the habit that some of the better forecasters in [an IARPA forecasting tournament called ACE] have gotten into: comparing their predictions to group averages, weighted-averaging algorithms, prediction markets, and financial markets.” See Ungar et al. (2012) for some aggregation-leveraging results from the ACE tournament.

Many forecasting experts add that when making highly uncertain predictions, it usually helps to decompose the phenomena into many parts and make predictions about each of the parts. As Raiffa (1968) succinctly put it, our strategy should be to “decompose a complex problem into simpler problems, get one’s thinking straight [on] these simpler problems, paste these analyses together with a logical glue, and come out with a program for action for the complex problem” (p. 271). MIRI’s The Uncertain Future is a simple toy model of this kind, but more sophisticated computational models — like those successfully used in climate change modeling (Allen et al. 2013) — could be produced, and integrated with other prediction techniques.

We should expect AI forecasting to be difficult, but we need not be as ignorant about AI timelines as we are today.

Acknowledgements