AI learns how to run by growing new limbs David Ha

What are the best two legs for running an obstacle course? One leg that crawls at the knee joint, and one massive leg dragged behind for stability like a kangaroo’s tail, according to a recent simulation.

David Ha, a researcher at Google, created a virtual robot with a wide head supported by two legs and tasked it with crossing a randomly generated landscape within a time limit. It learnt to do this with an algorithm used in artificial intelligence called reinforcement learning. As the AI moves through the terrain the algorithm rewards it with points for successfully supporting its head, reaching the exit, and not falling over. This reinforces good behaviour.

When the terrain is fairly flat, the AI crossed most quickly when it developed a jaunty skipping gait that it performed on the ‘knees’ of its long, skinny legs. This was more efficient than its previous attempts walking on the ‘feet’ of shorter, thicker legs.


When facing a landscape with stairs, pits and obstacles to climb over, the simulated robot developed one long, skinny leg that it used to sense upcoming obstacles, and one larger hind leg that it dragged in its wake to stabilise itself.

As is usual when you give AI free rein, hijinks ensued. When they gave the algorithm no constraints and asked it to cross an obstacle-laden course, the AI built an extremely tall bipedal creature that simply fell down to reach the exit.

When they rewarded it for building smaller legs, the AI took that to an extreme. Here it is tottering on tiny legs across a fairly flat surface.

In learning how to beat the game it is in, the algorithm also learned how to pick the appropriate size of limbs and how to use them. An evolutionary algorithm like this one could come up with designs we might not even think of that outperform our best options, wrote Ha in the paper describing the project.

Teaching an AI to follow rules on how to control its limbs while also building its own body takes patience and skill, says Oleg Klimov at research company Open AI in California “I salute David for doing that,” he says.

Reference: arxiv.org/abs/1810.03779