This essay considers, from a high-level design perspective, how the evolution of biological intelligence might inform our efforts to create general artificial intelligence. I address the questions of where a general AI agent should operate, what data the agent should work with, and what its reward functions should incentivise. The short answers are: in the real world, sensory inputs from its environment, and survival. This essay will explain why I think this will lead to general AI. I personally don’t have the expertise to put the concepts here into practice, so I’m putting this out there for anyone who is interested in building the actual thing.

This essay does not fully consider the technical challenges of creating a general AI; I’ll have to leave the technical work to real computer engineers. The biggest obstacle, it seems, is computational capacity. Given that physical reality is significantly more complex than the constrained data sets or virtual environments that most existing narrow AI operate within, I would guess that a reinforcement learning agent dealing with raw data from the real world would either take far longer or require more computational capacity to attain competency at a particular task. I don’t know of any existing unsupervised machine learning projects using raw real-world data. Autonomous vehicle AI like Tesla’s Self Driving and Waymo are currently being trained on real-world data, and they seem to be progressing reasonably well, but that is supervised machine learning because the agents are given examples of human drivers to mimic.

When considering what is general AI and how we might achieve it, we need to think of intelligence in terms of breadth versus depth. When comparing general AI versus narrow AI, I think what we mean is the breadth of intelligence. Narrow AI are narrow in the sense that, for example, the chess-playing agent cannot recognise images of cats, and vice versa. General AI should be able to do both and more, but not necessarily more competently than narrow AI. Competency refers to depth, for example which agent wins the most games of chess, given multiple agents that play chess differently from each other. We humans have broad general intelligence in that we can both play chess and recognise cats, but our chess intelligence is less deep than many narrow chess-playing AI. The problem of limited computational power, I would argue, relates more to depth than breadth. An agent that can learn to do a broad range of things is a general AI, even if it is significantly less competent at its broad range of tasks than the average human being. If we accept that broad but shallow AI qualifies as general AI, then limited computational power may not be a serious problem when attempting to build general AI; it just won’t be very smart at all the things it does.

Life-like motivations

To create general intelligence, we first need to identify the conditions that give rise to general intelligence. Our best known examples of general intelligence are, of course, us humans. To a lesser extent, other animals have also been observed to have some ability to solve novel problems, notably other primates, some mammals, and some birds. Why have we evolved this form of general intelligence?

I previously wrote a manuscript titled Love Letter to Our Future Robot Overlords, in which I argued that all life forms are driven by the need to survive and reproduce in a complex environment of finite resource availability and uncertainty. Animals must find food and avoid dangers to survive, and all our behaviours originated from this need. Note that these are open-ended objectives, in that there are multiple, possibly infinite, different ways to avoid dangers and find food. Some life forms, like plants and sponges, are able to find food without any intelligence at all. Insects are quite effective at finding food and avoiding dangers while having very limited intelligence, possibly less intelligence than some of our AI. As we progress to reptiles, birds, then mammals, we generally observe increasingly complex behaviours that allow the animals to find food and avoid dangers in an increasing variety of circumstances. What we call “intelligence” is an increasing amount of cognitive capacity applied to solving the problem of survival. We should note that animal intelligence, especially at the lower end, is more a result of evolved instincts than learning. However, in the sense that evolution through natural selection causes more successful patterns to be strengthened while less successful patterns are culled, its effects are similar to reinforcement learning.

In this schema, humans are somewhat unusual in that we have become so competent at survival that we have a lot of free time to do things other than finding food and avoiding dangers. To casual observation, it appears we pursue intellectual stimulation merely for the sake of it, reading books or watching movies or discussing politics. But such intellectual behaviour is also an extension of our survival impulse, because learning allows us to understand the world better, which allows us to solve more survival problems. General intelligence and intellectual curiosity is what happens when you give a life form, who is motivated to survive in a complex and uncertain world, a lot of seemingly excess brain power; it doesn’t necessarily need all this brain power to survive, but it can use it to survive more reliably and in a greater variety of circumstances.

As for love and hate, these are merely survival strategies in a social context, when interactions between multiple people are possible. Consider if you are a prehistoric human, without much access to tools or technology. There is a tall fruit tree and you need a friend’s help to climb into its branches to reach the fruit. Once you are up in the tree, you might as well collect some fruit for your friend as well, so they will be willing to continue helping you in the future. You need each other’s help to survive, so you form a long-term, cooperative friendship. In contrast, consider if another person attempts to steal your food, and it is less costly for you to fight this person than to give up the food. This obviously motivates conflict. Thus, depending on whether cooperation or conflict helps you survive in a particular social situation, you are driven to love or hate. We would consider such behaviours to be indicative of general intelligence, and it results from our need to find food in a complex social environment.

To summarise, here are the conditions that have given rise to our general intelligence:

Life forms that require sustenance from the environment to survive.

Challenges in the environment that hinder the life forms’ survival.

A complex environment where there are multiple (or infinite) ways for life forms to acquire sustenance, and multiple (or infinite) ways to overcome environmental challenges.

Minds that are able to learn or evolve, and which are inherently motivated to survive.

Enough cognitive capacity for complex behaviour.

Broad questions, broad answers

At present, we have invented some very effective narrow AI, but we do not yet have convincing general AI. For narrow AI, consider the work of Boston Dynamics or DeepMInd’s AlphaGo. The Boston Dynamics robots have become very good at locomotion, but they only go where you tell them. AlphaGo is clearly superhuman at playing go, but it doesn’t do anything except play go. In contrast, we imagine that a general AI would have a much broader understanding of the world and a much wider range of capabilities, although they may be less competent at specific tasks than narrow AI. Are we lacking some technological breakthrough that prevents us from developing general AI, or are we simply not asking AI to do the right things? Boston Dynamics reward their AI for solving locomotion, and DeepMind reward AlphaGo for winning games of go. We have not, as far as I am aware, tried to reward AI for finding sustenance in the real world, which is what gave rise to our own general intelligence.

Note that locomotion, winning go, and indeed most applications of AI are problems where one converges onto a solution. Consider locomotion: for any given robot body and any given environment, there is one mechanically optimal way to move from point A to point B. This means, if you reward an agent for optimising locomotion from point A to point B, it will converge on only one answer. Winning go games is somewhat more complicated because of the enormous number of possible moves in a game, but in each turn of the game AlphaGo is trying to find the single move that has the highest probability to lead to a victory — again, one answer at a time. When your reward functions are given for finding the single best answer to something, the agent will end up doing only that one thing. AI are narrow at least partly because we ask them to be narrow.

In contrast, the survival imperative of a biological life form is an open-ended problem with many (possibly infinite) possible answers. More intelligent minds with more knowledge can find better ways to survive, which is why evolution has created increasingly intelligent life forms over time. Consider our need to find food, if I were a prehistoric human. Perhaps I go foraging, and I may go to the forest or the beach. Perhaps I hunt, and I may chase game over the fields or dig animals out of their burrows. I may go trapping, or go fishing with a line or a net. I may grow crops or domesticate animals. I may take food from someone else, either through trade, theft, fraud, or robbery. All these possible courses of actions involve different behaviours, complex environments, and uncertainty. In most cases, being smarter makes me more effective at whatever I attempt. Thus, a simple cost function of hunger can motivate me to learn and attempt a very wide variety of behaviours. We have general intelligence because our survival in a complex environment incentivises general intelligence. That is not to say that general intelligence is inevitable, because many simple life forms pursue only one way to survive, for example the earthworm simply tunnels forward eating whatever it encounters. But the earthworm’s narrow intelligence results from its very limited nervous system, not because it’s survival imperative is any different from ours; give it a bigger brain, and it will likely behave in more interesting ways.

If we want general AI, perhaps we should similarly reward them for surviving in a complex environment, and then give them plenty of computational capacity.

Anatomy of a living machine

If complex behaviour develops only in response to a complex environment, then the best place to train general AI is in the real, physical world. Much AI research is done in virtual environments or on data sets because these are lower cost and easier to control, but when the agents optimise themselves for unrealistic environments, we should expect them to exhibit unrealistic behaviour. Biological life forms behave the way they do because they are constrained by the laws of physics, and the most complete environment for experiencing the laws of physics is the physical world. Thus, the first requirement is that our agent should control a physical, robotic body that interacts with the real world.

There are many different ways to design a robot to draw sustenance from its environment. In my opinion, the most effective system, in terms of simplicity of design versus complexity of possible behaviours, is to have a photovoltaic powered robot with a battery. If such a robot is controlled by a reinforcement learning agent with a reward function for keeping the battery charged (or a cost function for letting the battery drain, or both), the agent should learn to move the robot toward and stay in the light, not unlike many sunbathing cold-blooded animals. Given the complexities of the real world, the availability of light is affected by day-night cycles, the weather, seasons, artificial lighting, other life forms, and many other factors. The more it is exposed to and affected by these environmental factors, the more complex and nuanced its light-seeking behaviour can become.

The idea of a light-powered robots is not original, of course; existing examples range from simple toys to Mars rovers. The novelty of what I’m proposing here is to give a light-seeking robot a far greater reinforcement learning capacity than it should ever need, and then to put it in an environment that is deliberately complicated, where there are always better and more intelligent ways to solve the problem of how it gets light.

The specific reward functions for when and how much the agent should charge the robot’s battery are similar in principle to our feelings of hunger and satiety. A person begins feeling hungry if they haven’t eaten for perhaps 4–8 hours, but actually a person can survive for more than a month without food (don’t try this at home). Our robot will need enough battery life to last at least through the night, if it is subjected to a natural day-night cycle. Even with a 24 hour battery life, it will need to charge itself to above maybe 60% of full charge before nightfall to remain active through the night. It is possible that the agent might learn to “sleep” to conserve energy at night, or else park the robot where it expects the sun to be available in the morning, allowing the battery to drain completely overnight, to recharge automatically when the sun rises. When the battery is full, the agent should probably not “feel hunger” as soon as the battery starts draining, or else it becomes a disincentive for any sort of exploration or experimentation. Indeed, a fully charged battery may invoke a cost function, equivalent to us feeling overly full when we eat too much. The optimum specific thresholds for when the reward functions take effect will depend on the capacity of the battery, how quickly it charges, and how quickly it drains with activity. At a high level of charge, there should be no reward or cost. Draining the battery should activate increasingly severe cost functions, while recharging the battery should be rewarded.

There are other possibilities for powering a robot, but I think these are less effective for our purposes. The easiest way to power any machine is to plug it into the electricity grid, but giving the agent this option, or easily accessible charging stations, would greatly reduce the challenge of finding sustenance, and therefore reduce the complexity of its behaviours. This may be an option if we simply don’t have enough computational power for the agent to learn the relationship between sunlight and battery charge, though I’m not sure how that is more complicated than the relationship between charging stations and battery charge. Another source of easily accessible energy is biological matter, whether this is somehow “digesting” other life forms, perhaps by converting them into biofuels, or combusting dried plant matter to extract concentrated heat energy. It is conceivable that a robot may have a portable distillery in its body to convert biological matter into fuels, but this seems much more complicated than simply having a photovoltaic panel. Also, it seems like a dangerous idea to deliberately design an autonomous robot that may directly compete with us for food, or perhaps even eat us for food.

The robot will need a way to move around, and to interact with its environment. Legs seem more appropriate than wheels if we’re interested in complex behaviour, allowing climbing and jumping. One or more prehensile limbs will enormously expand the robot’s range of possible behaviours, but these include more destructive and dangerous actions, so perhaps this option should be avoided until the agent’s behaviours are better understood and sufficient safeguards are developed. The robot will likely benefit from a broad range of sensory devices, including cameras, directional microphones, temperature and touch sensors, accelerometers, gyroscopes or some other way to determine the direction of gravity, force gauges for all joints in its body, and perhaps a humidity sensor. While the camera is obviously useful for seeking out light sources, a wider range of senses will facilitate more complex interactions with its environment. Perhaps, if we are concerned about excessive complexity, we begin with only a few senses, and add more over time. Of course, the robot will also need to be able to sense how much charge remains in its battery.

The physical features described above are somewhat similar to Boston Dynamics’ Spot robot, but there are key differences and challenges. Importantly, Spot does not have much space to mount a photovoltaic panel, and its battery life is only 90 minutes. This suggests that our robot will have to be flatter and wider to have enough space for a bigger battery and photovoltaic panel, and perhaps it should have a lower centre of gravity and a more deliberate gait, so that its movements are more energy efficient. In other words, something less like a dog and more like a turtle or beetle.

Unlike biological life forms, the robot will not self-repair damage or reproduce. However, there is still the problem of what to do when the agent does something that breaks the robot. The simplest answer would be for humans to repair the robot whenever it gets damaged, but the agent may adapt to this in ways we don’t want. For example, if the agent learns that humans will always repair the robot, it might become reckless and take unnecessary risks, or it might deliberately break itself to have old but still functional hardware replaced. The reason animals are careful to avoid injury is because healing incurs material and energy costs, not to mention the pain and temporarily reduced capabilities. Externalising the costs of injuries through human intervention would likely encourage more risk-taking. Alternatively, the robot’s environment might include a range of spare parts that it can use to repair itself, such that “healing” requires the agent to spend time and energy, although I’m not sure how it would learn to do this if that behaviour wasn’t pre-programmed. If the agent becomes smart enough, it might learn to use the parts to redesign the robot it controls, but we would be well into general AI territory by that point. Perhaps the most suitable solution, at least to start with, will be for humans to repair the damaged robot, but also add a significant cost function whenever it happens.

Let’s say we do all of the above: we have a robotic photovore equipped with a variety of sensory devices, controlled by a reinforcement learning agent whose reward and cost functions motivate it to keep the robot’s battery charged. Then what?

Learning to live

The basics

The first things we need the agent to learn are: 1) the relationship between light and the amount of charge in the robot’s battery, and 2) how to move the robot around. Let’s say we put the robot in a simple environment of an empty room with a small window facing the sun. Each day, the sun rises and shines through the window, creating a patch of bright light on the floor. As the sun arcs across the sky during the day, the patch of light moves, until the sun sets in the evening. Then, at night, it’s dark. This creates for the agent a variable resource it can use to charge the robot’s battery, but the agent must move the robot to stay in the patch of light, and at night it is “stressed” by the absence of light, which it must wait out. In the morning, when the light returns, the agent should move the robot back to where the light is. This simple environment incentivises the agent to learn how to move the robot towards the light.

The benefit of this setup is that it is very simple and natural. The problem is that it might take the agent a very long time to learn how to sunbathe, because of how slowly the sun moves, and how slowly the battery charges and drains. The agent will need to experience the sun moving onto and away from the robot’s photovoltaic panels multiple times before beginning to identify the correlation between the light and its battery charge. That could possibly take months. We might try to speed up the process by replacing the window with a moving spotlight, but the agent’s reward functions are not for following the light, but for charging the robot’s battery. The key factors here are how much charge the battery can hold, how quickly it charges in the sun/spotlight, and how quickly it loses charge in the shade. These factors must be balanced so that being out of the spotlight will cause the battery to lose its charge at a fast enough rate that the agent has some urgency in moving back into the light, but moving should not drain the battery so much that being in the light won’t recharge the battery enough to compensate. Even if all these parameters can be successfully balanced so that the best way for the agent to keep the robot’s battery charged is to move around to follow the spotlight, you’ll end up training the agent for an artificial environment where its “food source” moves around very quickly, and there’s no guarantee this learned behaviour will help it once it is placed back to a natural sunlight scenario.

The other major challenge is getting the agent to learn how to make the robot walk, and to extract useful information from its camera feed. Unlike the robots of Boston Dynamics, our photovore isn’t walking for the sake of walking, but walking to move its photovoltaic panels into the light. This turns movement into a second-order objective, one that it must first solve before it can attain the rewards for moving into the light. Likewise, the agent must also learn to identify where the light is using the robot’s camera, before it can move the robot to it. We can, and we probably should, short-cut this process by pre-programming the robot with certain “reflexes”. We know that, in the natural world, many newborn animals automatically start using their limbs to move around as soon as they are strong enough to do so (a newborn fawn will almost immediately, though shakily, try to stand up on its legs), and they will also automatically seek food (baby mammals will move to suckle from their mothers, bird hatchlings will open their mouths to be fed by their parents, etc). Likewise, we know our own vision system incorporates some automatic image processing, including image stabilisation, colour correction, and edge detection. Instead of giving the agent direct access to the robots limb controls, we might programme the robot with basic walk cycles, and allow the agent to activate these instead. Instead of the direct camera feed, the agent might be given a processed image feed, perhaps one that highlights depth-perception cues, areas of intense light, and other important information. Perhaps, beyond being rewarded for charging its battery, it should also be rewarded for moving towards light, at least initially. Giving the agent these “unconscious instincts” should greatly reduce the difficulty of it learning to move the robot into the light.

However, we are ultimately interested in complex behaviour, including novel movement. If the agent will ever learn to make the robot run and jump and do backflips, it will also need to have direct control of the robot’s legs, not just pre-programmed walk cycles. Having access to the raw, unprocessed camera feed may also provide it with useful information in a future scenario that we cannot anticipate. Within our own minds, we humans have the ability to override some of our instincts. For example, we usually recoil from pain, but we can also force ourselves to endure some amount of pain if we want to. Thus, the agent should also benefit from having access to both the pre-programmed functions and direct, unfiltered access to the robot’s capabilities. It is possible that we give the agent access to the pre-programmed functions first, then introduce direct control only after it has mastered the basics. When it becomes competent with direct control, we may remove the pre-programmed “instincts” altogether. This is likely important if we initially reward it for moving towards light; while that “instinct” will be useful at the beginning, it can also be an impediment to more complex behaviour, for example it might disincentivise the agent taking an indirect route to reach the light.

We want our photovore robot to do a lot more than just navigate an empty room. Once it can do that with some degree of competency, the next step is to add obstacles and other objects into the room for the robot to interact with. The light continues to move from one side of the room to the other over the course of the day, and the agent wants to make the robot follow the light. This sets the stage for the agent to learn complex movements and problem-solving. Consider if we put a low wall across the room, and have stairs up and down each side. Or perhaps the stairs are movable, but they are scattered about the room, and the robot must push the stairs into place. Perhaps there are pathways blocked by loose debris, or a deep trench that the robot must jump across to reach the light. Solving such navigation problems does not seem very different from getting a robot to move from point A to point B, but the key difference is we don’t reward our agent for moving to specific locations; we are rewarding the agent for doing whatever it takes to charge its battery. This means it can potentially learn to do much more that just walking to the light.

Cooperation

I promised love and hate in the title of this essay, so let’s move on to social behaviour. Social behaviour requires more than one participant, so we first make a copy of our agent and robot. Then we put the two of them into an environment where each robot can only get to the light with the other robot’s help. Consider if we have a transparent wall across the room, and a door in the wall that is opened by buttons on each side of the wall. To get through the wall, one robot must hold the button while the other moves through the door. The optimal cooperative strategy for both robots to get to the light on the other side of the wall is for one to hold the door open for the other to pass through, then for that robot to hold the door open from the other side for the first robot to pass through also.

First, the agents have to learn the relationship between the button and the door; perhaps we do this part before we make the copies. Then, the agents need to learn that holding the button helps the other robot get to the light, but not itself. I would guess that, in the beginning, one agent will, possibly accidentally, open the door for the other, but the other would just go lay down in the light, and not help its compatriot; this is because there’s no reward function for helping the other. But because the agent that opened the door gets left in the dark, it receives costs for not getting to the light, so the next day it wouldn’t open the door. Perhaps it waits for the other robot to open the door, then it moves through and go lay in the light. Over time, each agent finds that the other is uncooperative unless the other is also able to get to the light, at which point they will learn to help each other.

There is significant optimism in assuming the agents will solve this scenario without help. For one, social animals like us have many unconscious social emotions and instincts, including empathy and gratitude and guilt, such that we feel good when we help others, and we feel bad when we hurt others (or most of us do, at least). These instincts motivate us to start helping each other even when we don’t really know how it’ll benefit us. Once again, we can provide our agents with some shortcuts to facilitate cooperative behaviour. We might allow each agent to sense the battery charge of the other, either directly or through some visual indicator on the robot’s body, and we might give the agents a small reward function for keeping other robots charged; this would be analogous to our empathy. Logically, these short-cuts shouldn’t be strictly necessary, because in this scenario the only way each agent can reliably convince the other to cooperate is to make sure both robots can access the light. But if each agent cannot directly sense the other robot’s battery charge, and if there is no built-in “empathy”, it might take a very long time for the agents to find the optimum strategy starting from random actions, or they may not find it at all due to limited computational capacity.

There are many other ways to design environments where cooperative behaviour is necessary for the agents to reach the light. We might have a wall that is tall enough to require them to climb over each other to reach the other side; that would test not only cooperation, but also novel movement, especially if the robot at the top of the wall learns to reach down and pull the other up. We might have more elaborate puzzles that require more than two robots to solve. We might design puzzles that can only be solved through tool-making. We might even just put the robots in randomly generated environments and see what happens. The possibilities are endless.

At some point, the agents may invent a rudimentary language to facilitate their cooperative problem-solving, but such emergent behaviour is difficult to predict.

Competition

As the robots interact in complex environments in an unconstrained manner, we may begin to see competition between the agents, if one tries to force another to help them, without reciprocating. But if we really want to teach our agents violence, the most effective way is likely by introducing resource scarcity. We know that, in the real world, once animals start thinking maybe there isn’t enough food to keep everyone alive, at least some individuals will start fighting for control over the scarce resources. Up until now, if the agents can move their robots into the light, they can keep their batteries charged. If we reduce the size of the window until the patch of light is too small to keep all the robots charged, then some of them will have to “die”, their batteries depleted. The question is: who dies? And how do they decide? Each agent wants to move into the light, but if there is only enough light to charge one robot, they will all try to occupy the same space at the same time. The expected result is physical conflict. If, at first, some of them just give up without much of a fight, we can always increase the size of the window again, let them recharge and revive, then reduce the window size and try again. Eventually, they will learn that overpowering the other robots and keeping others out of the light will allow them to gain the reward.

If all the robots are equal in physical capabilities, such a free-for-all, winner-takes-all scenario may remain unstable because who eventually wins the right to sunbathe is more or less a matter of luck. Instead, we may make one of the robots physically stronger than the others, or perhaps some of the robots have unintentional manufacturing defects. This changes the scenario, where some of the robots become more likely to win fights than others. Over time, the agents controlling the weaker robots may learn that fighting is an ineffective expenditure of battery charge, and cede the sunbathing rights to the stronger robots. This would lead to the emergence of a basic “social hierarchy”.

Alternatively, instead of reducing the light availability to charge only one of the robots, we may have a less severe restriction, such that most of the robots can be fully charged if a minority gets no sunlight, or else all the robots can be partially charged by sharing the limited sunlight. In such a scenario, two different outcomes are possible. Either a majority of the agents decide to gang up on a minority of the robots, starving the minority to feed the majority, or they decide to share, everyone goes a little hungry, but nobody starves. I can’t predict what will happen then.

Again, we may help the agents arrive at competition and violence more quickly by rewarding them for behaviours similar to our greed or vengeance or bigotry. But this seems like something we will deeply regret if the robots ever get out of control.

So, this is what I mean by teaching robots to love and hate. First, you must create robots, controlled by reinforcement learning agents, whose survival depends on gathering a scarce resource from the environment, and you reward them for doing so. Then you must put multiple copies of them in an environment where either cooperation or competition is necessary for them to access this resource. Given enough time and computational capacity, they will learn to help or harm each other.

Towards general artificial intelligence

Having AI learn to help or fight each other is impressive, but I still don’t think that is general AI. I said earlier that if you reward an agent for converging onto one optimal solution, it will end up doing only one thing, and this is a narrow AI by definition. I argued that survival is an open-ended problem that requires general intelligence to solve, but so far I have only described relatively simple environments where there is a single optimal solution. The room has a patch of sunlight. Whatever action gets the robot into the sunlight most reliably, and expending the least energy, is the optimal solution. This is assuming the agents haven’t done something completely unexpected yet, such as breaking out of the room (there is, after all, much more sunlight outside the window).

In order to nurture general AI, we must put the robots into a much more complex environment, and that means letting them loose into the real world, out of their little room. Remember that the reward function is for keeping the battery charged, not necessarily for sunbathing. There are many other ways in the real world for finding light, and for charging batteries. There are also many more obstacles that can drain the robot’s battery or prevent it from charging. Once the robot is out in the real world, the possibilities broaden dramatically, to the point where the agent cannot reliably keep the robot charged without general intelligence. The agent will have to learn to deal with rain and snow and dust. It will have to avoid environmental hazards, animals, and troublesome people. Consider if the agent is able to control the robot across long distances, perhaps over a satellite-based internet connection. The robot might climb a mountain to catch earlier sunrises and later sunsets. It might travel to a desert where there are no clouds and no rain, and therefore more reliable sunlight. It might go to a busy city where there are bright lights at night. It might learn to build a bonfire at night, to sit in the light of the fire instead of being in the dark (hopefully it doesn’t learn to burn down people’s houses every night). It might migrate across the equator each year to chase the longer summer days, the same way migratory birds do. It might somehow find its way past the Arctic Circle or down to Antarctica, where the sun never sets in summer, though it would regret that decision if it doesn’t leave before winter sets in. It might get itself launched into space to orbit the sun directly. I realise many of these ideas are quite fanciful, and they would require our agent to be far more intelligent than any AI we know today, but my point is that a simple incentive of keeping its battery charged, if given enough time and intelligence, can lead to all these behaviours. Survival is an open-ended problem, and the solutions are limited only by the available computational power.

Actually, the most likely outcome, I imagine, is the agent will learn how to wire a power supply and power plug to the robot’s battery, and plug it into a wall socket, keeping the battery charged regardless of light or dark. But even that means the robot understands enough to upgrade itself. Singularity, here we come.

A side note on AI understanding semantics

The crux of this essay is that, if you want AI to understand the real world, it needs to interact with the real world as directly as possible. At present, there are unresolved obstacles with getting AI to understand things like the meaning of text or identifying things in images. Consider that the narrow AI agents have never interacted with the things described by the text or shown in the images. Their entire existence is just text or images. If I say to you, “cat”, you imagine the living, breathing, moving, furry, meowing animal. If you say to an image recognition algorithm, “cat”, it thinks about the thousands of still images that are labelled “cat”. Of course the agent doesn’t understand what a cat is; it’s never seen a real cat, and it thinks “cat” means a whole lot of still images. But, I’ll bet you, if our photovore robot, who moves around in the real world, spends enough time interacting with real living cats, it will eventually learn to recognise cat pictures better than the image recognition algorithm. It might even learn human languages, because it is in a position to see that the words are not things in themselves, but they are signs referring to real things, and it will have interacted with these real things. An algorithm whose only data input is text will never understand this.