OpenAI previously demonstrated an algorithm capable of competing against top humans at single-player Dota 2. The latest work builds on this using similar algorithms modified to value both individual and team success. The algorithms do not communicate directly except through game play.

“What we’ve seen implies that coordination and collaboration can emerge very naturally out of the incentives,” says Greg Brockman, one of the founders of OpenAI, which aims to develop artificial intelligence openly and in a way that benefits humanity. He adds that the team has tried substituting a human player for one of the algorithms and found this to work very well. “He described himself as feeling very well supported,” Brockman says.

Dota 2 is a complex strategy game in which teams of five players compete to control a structure within a sprawling landscape. Players have different strengths, weaknesses, and roles, and the game involves collecting items and planning attacks, as well as engaging in real-time combat.

Pitting AI programs against computer games has become a familiar means of measuring progress. DeepMind, a subsidiary of Alphabet, famously developed a program capable of learning to play the notoriously complex and subtle board game Go with superhuman skill. A related program then taught itself from scratch to master Go and then chess simply by playing against itself.

The strategies required for Dota 2 are more defined than in chess or Go, but the game is still difficult to master. It is also challenging for a machine because it isn’t always possible to see what your opponents are up to, and because teamwork is required.

The OpenAI Five learn by playing against various versions of themselves. Over time, the programs developed strategies much like the ones humans use—figuring out ways to acquiring gold by “farming” it, for instance, as well as adopting a particular strategic role or “lane” within the game.