http://journals.cambridge.org Downloaded: 21 Apr 2014 IP address: 163.1.72.155

WHY WE NEED FRIENDL Y AI

Luke Muehlhauser and N ick Bostrom

Humans will not alwa ys be the most intelligent

agents on Earth, the ones steering the fu ture. What

will happen to us when we no longer play that role,

and how can we prepar e for this transition?

The human level of intelligence is an evolutionary acci-

dent – a small basecamp on a vast mountain side, far

below the highes t ridges of intelligence allowed by physics.

If we w ere visited by extr aterr estrials, these beings would

almost certainly be very much more intelligent and techno-

logically advanced than we are, and thus our future would

depend entirely on the content of their goals and desires.

But aliens are unlikely to make contact anyti me soon. In

the near term, it seems more likely we will crea te our intel-

lectual successors. Computers far outp erform humans in

many narrow niches (e.g. arithme tic and chess), and there

is reason to believe tha t similar large impr ov ements o ver

human performa nce are possible for general reasoning and

technological development.

Though some doubt that machines can possess certain

mental properties like consciousnes s, the absence of such

mental properties would not prevent machines from becom-

ing vastly more able than humans to efficiently steer the

future in pursuit of their goals. As Alan T uring wrote, ‘ ... it

seems probable that once the machine thinking metho d

has started, it would not take long to outstrip our feeble

pow ers .. . At some stage ther efore we should have to

expect the machines to take control ... ’

There is, of course, a risk in passing control of the future

to machines, for they may not share our values. This risk is

increased by two factors that may cau se the transition from

doi:10.1017/S1477175613000316

#

The Roya l Institute of Philosophy , 2014

Think 36 , V ol. 13 (Spring 2014)

Think Spring 2014

†

41