This Part 3 of a series on the five- to twenty-five year future of smart agents and their knowledge bases. Each post explores a different aspect of the agent-enhanced future. The series focus is personal AIs, personalized agents that will increasingly aid and advise us, and act as our proxies in the world. Our children and friends will interact with our PAIs when we are gone. We will come to see them as intelligent extensions of ourselves. Because of this, I think PAIs, agents and their knowledge bases are the most important IT development we’ll see in the next generation. Agree? Disagree? Let me know.

Let’s begin by zooming out to a universal look at the amazingly rapid change we are seeing all around us, and the nano- and info-acceleration that drives this change. That will help us see where we are going, to both physical and virtual “inner space.” Then we’ll look at mediated reality, one of many labels for our increasingly virtual environment. These two topics should help us understand, in broad strokes, where most of us will increasingly “live” in the 21st century. Hint: It may not be where you think.

Our Great Race to Inner Space: Seeing Nano- and Info-Acceleration

“World in a Buckyball.” Image by nanoscientist Chris Ewels (2003).

We live in an incredibly special place and time. Our computers are learning to map, simulate, and “see” the world around them, and now even to think, in increasingly human-competitive ways. They are also accelerating in these abilities, because of the special nature of our universe. Scholars and future-thinkers are coming to recognize that just two of our technologies, nanotech and infotech, continually accelerate. Because of these two techs, humanity is presently engaged in a Great Race to “Inner Space”. Our next frontier isn’t “Outer Space” — doing things at human scale, or venturing to the stars — but rather “Inner Space”, porting all the processes and objects we care about into both nanotech (physical inner space) and infotech (virtual inner space). We do this because physical inner space (nanotech) continually delivers results that are “Faster, Smaller, and Cheaper”, and virtual inner space (infotech) continually delivers results that are “Smarter, Stabler, and Better”.

Surprisingly, when any other process on Earth accelerates, it is always a secondary acceleration, a direct consequence of the primary accelerations of these two very special technologies. All biotech accelerations, for example, occur only via accelerating nano and infotech advances. All social accelerations, when anything “goes viral,” are driven by exponential replication of information, in brains or machines. Because of this fact, understanding these two techs is now critical to strategic foresight.

Let’s look first at nanotech. Ever since human civilization emerged, we have been drilling down ever smaller scales of observation and activity, and we’ve done that faster every year. In just under 90 years, between 1895 and 1983, the smallest distances accessible to science shrank a hundred million fold, and with quantum computing, we’re journeying further inward still. See Abraham Pais’s majestic Inward Bound (1988) for that amazing story.

What’s more curious, every time our engineers and scientists have figured out how to do things on smaller and denser scales (make energy, sense, produce, filter, store, compute, communicate, etc.) their processes become stunningly more capable and more efficient. Not just a bit more capable and efficient, but orders of magnitude more, often in a single innovation step.

Advances on the human scale typically give us 10%, 30%, 50%, sometimes even 500% (5X) improvements in a single step. Think of Ingvar Kamprad’s famous invention of flat-packed furniture. That one idea gave IKEA a roughly a 5X increase in shipping efficiency. But innovations at the nanoscale often start at 10X (1,000%) capacity or efficiency improvement, and routinely go as high as ten millionfold (10,000,000X), in a single step.

Consider a few examples: We split the atom in the 1940s, and we got 1,000X more energy yield per mass in response. Now we’re trying to fuse atoms (fusion). When we do this, by mid-century perhaps, we will get 1,000X more energy yield per mass yet again. When a chemist adds a functional group into the active site of an enzyme, she often gets a 1,000X increase in speed, yield, stability, or efficiency. A microlaser called a photonic crystal, discovered by one researcher in 2005, is 1,000,000X more energy efficient than previous microlasers. And so on. See any good book on nanotech, like Boysen and Muir’s excellent Nanotechnology for Dummies, 2nd Ed. (2011), or futurist Eric Drexler’s latest, Radical Abundance (2013), for many more such examples.

Why does this matter to our series? Because the hardware on which tomorrow’s agents will be built is seeing vast nanotech improvements every year, due to the current global explosion of smartphones and interactive entertainment (video games). Nvidia’s Pascal GPU, being released next month, has 5X faster connections to memory than previous GPUs, and uses 2X faster floating point math. Together these will improve the performance of neural networks and deep learning (our next post) by 10X. In a single product step. In general, hardware based neural nets are more than 100X faster than software neural nets. They are also up to 1,000,000X more energy efficient. Optical neural nets, which we will eventually build, are over 100,000X faster than electron-based neural networks. And so on. The further we venture into nanotech, the more amazing performance and efficiency gains we get. That’s how the universe works, whether we want it to or not.

The second special technology is infotech. Whenever humans or machines increase their computational ability, they can use that new ability, in a virtuous cycle, to make smarter, stabler, and better things. Part of why they are better is that the more things become information-enabled, the less physical resources we need. We substitute information — and intelligence — for physical activities and things, a process called “dematerialization.” Think of all the physical actions and devices that an iPhone and its apps replaces, and makes unnecessary. Think also of the ever-growing fraction of our economy based solely on bits, not atoms. As infotech advances, the world itself becomes not just physical, but increasingly virtual.

Systems that use nano- and info-acceleration are special, because they can increasingly get around physical resource limits in doing their jobs. They run into resource blocks, but the smarter they get, the faster they find a new way to use nano and infotech to get around them. I’ve studied these exponential technologies for over a decade now, and founded a small nonprofit to study them, the Acceleration Studies Foundation, in 2003. These accelerations aren’t going to end anytime soon. Because of the physics of our universe, the smallest scale of spacetime that we know of, the Planck scale, is a long way from where nano and infotech reside today. If you’re wondering where the acceleration ends, I speculate on that in a 2011 paper, the Transcension Hypothesis. Here’s a lovely two-minute visual intro to the hypothesis by futurist Jason Silva.

Whether this hypothesis turns out to be true or not, so far, complexity’s history has been a Great Race to Inner Space, not Outer Space. Every leading system, starting with large scale universal structure, then galaxies, then life-supporting planets, then bacterial life, then multicellular life, then social vertebrates, then humans, and now our self-learning computers, has emerged via an accelerating journey inward, to ever greater physical and virtual inner space.

Each Human Brain Contains at Least 80 Trillion Unique Synaptic Connections

Which systems on Earth have gone the furthest into inner space so far? That would be human thinking and consciousness. They are the most advanced computational nanotech (physical inner space) and infotech (virtual inner space) we can presently point to. Consider that there are 80 trillion informationally unique synaptic connections inside every three-pound human brain — an incredible feat of nanotech. All our moral, empathic, and self-, social-, and universe-reflective thinking and feeling are virtual realities (infotech) that has arisen directly from that nanotech. Yet tomorrow’s machines will venture even further and faster into nano and infospace.

Consider how the knowledge web, just twenty-seven years after its invention in 1989, is already a vast, low-level simulation of physical reality, even before high-level VR, AR, and simulations have arrived. We’ve also seen accelerating virtualization (simulation) of hardware, operating systems, infrastructure, and business processes since the 1960s, and ever more of what we do goes into the cloud every year. As entrepreneur Marc Andreessen said in 2011, “software is eating the world.” Most curiously, as we’ll see in our next post, the most advanced simulations being built today are moving from being “artificial” intelligences, specified top-down by human engineers, to natural intelligences, emerging bottom-up, with us as trainers and “gardeners”. These machine intelligences seem destined to be the next natural form of mind and self-awareness on Earth.

To recap, when we think carefully about it, we must admit that thinking and consciousness, whether in humans or machines, are virtual worlds. They are as real in the universe as the physical world. That’s why we should stop using the phrase “‘real’ world” as an opposite to “virtual world.” Information is as real as physics in the universe we live in. All life’s virtual processes emerge out of physical reality. So do the realities our computers are creating for us. So rather than “real” and virtual, physical and virtual — or nanotech and infotech — are the right pair to think about as we run this Great Race.

Hopefully, this helps us appreciate why our emerging agent hardware and software are so special. Tomorrow’s agents will use both nanotech, and biologically-similar infotech, like deep learning, to accelerate their emergence. As their capabilities grow, we’ll use them to drive many secondary accelerations, including human creativity, wealth production, and mass-desired social changes like basic income. They’ll also drive the elimination of disease and involuntary death. These are big claims, and we’ll explore them further in future posts. Meanwhile, please let me know if you disagree.

Mediated Reality: Our Increasingly Virtual, Digital, & Intelligent Planet

We have talked previously about the semantic aspects of the knowledge web, but that’s just one of its virtual aspects. Let’s look this rising virtualization more broadly now.

As with agents, several terms are presently bandied about to describe our emerging shared virtual space. The metaverse is one. Mixed reality and simulated reality are others. Artificial reality is another. My favorite term to refer to our increasingly virtual, digital, and intelligent infotech is mediated reality, short for computer-mediated reality. This phrase draws attention not just to the simulation abilities of our infotech, but to its growing intelligence, and to what is happening to us as digital mediation grows. Our infotech is progressively mediating, or acting as an intelligent interface to, almost every aspect of our physical environment.

Four Axes of The Metaverse Roadmap (Smart, Cascio & Paffendorf, 2007)

In 2007, I co-authored an industry study on our virtual future, The Metaverse Roadmap. We divided virtual space into four domains: augmented reality (AR and VR), virtual worlds (games and social worlds), simulations (“mirror worlds”) and archives of human activity (“lifelogs”). Let’s look at each of these now, and some of their implications for agent emergence.

Augmented Reality and Virtual Reality

AR will increasingly mediate our lives in coming years. Visual AR gets the most attention, as it is so flashy, and we’ve seen impressive recent demos of AR games from Magic Leap and Microsoft’s HoloLens, as in the image below.