Last month, several members from the AI @ Unity team were present at NeurIPS in Montreal. At the Unity booth, we had the opportunity to meet hundreds of researchers and introduce them to Artificial Intelligence and Machine Learning projects at Unity. Later this month, we’re heading to AAAI-19 (an annual AI conference) in Honolulu where we’ll be hosting a booth, and also co-organizing the AAAI-19 Workshop on Games and Simulations for Artificial Intelligence. In this blog post, we’ll provide you with a brief overview of the workshop and explain why we are eager to foster research that leverages games and simulation platforms.

If you’re attending AAAI, consider joining our workshop on January 28 – it’s packed with fantastic speakers and papers covering games and simulations for AI. Also, drop by our booth (January 29 – 31) to say hi, watch some demos, and learn about teams and projects at Unity.

A brief history of games in AI research

Games have a long history in AI research, dating back to at least 1949 when Claude Shannon (shortly after developing information entropy) got interested in writing a computer program to play the game of Chess. In his paper “Programming a Computer for Playing Chess”, Shannon writes:

“The chess machine is an ideal one to start with, since: (1) the problem is sharply defined both in allowed operations (the moves) and in the ultimate goal (checkmate); (2) it is neither so simple as to be trivial nor too difficult for satisfactory solution; (3) chess is generally considered to require “thinking” for skilful[sic] play; a solution of this problem will force us either to admit the possibility of a mechanized thinking or to further restrict our concept of “thinking”; (4) the discrete structure of chess fits well into the digital nature of modern computers.”

That was in 1949. Since then, there has been an enduring interest in creating computer programs that can play games as skillfully as human players, even beating respective world champions. Shannon inspired Arthur Samuel’s seminal work on Checkers in the 1950’s and 1960’s. While Samuel’s program was unable to beat expert players, it was considered a major achievement as it was the first program to effectively utilize heuristic search procedures and learning-based methods. The first success story of achieving expert-level ability was Chinook, a checkers program developed at the University of Alberta in 1989 that began beating most human players and by 1994 the best players could at best play to a draw. This trend continued with other 2-player board games such as Backgammon (with Gerald Tesauro’s TD-Gammon, 1992-2002) and Chess (when IBM’s Deep Blue beat Garry Kasparov, 1997), and most recently with Go. An important scientific breakthrough of the last few years was when, in 2016, DeepMind’s AlphaGo beat 18-time world champion Lee Sedol 4 to 1, the subject of the Netflix documentary, AlphaGo.

(Source) Chinook vs Marion Tinsley (1994)

The progress over the last 70 years since Claude Shannon’s paper has not been limited to solving increasingly more difficult 2-player board games but has expanded to other complex scenarios. These include 3D multiplayer games such as Starcraft II and Dota 2 and more challenging game tasks such as learning to play Doom and Atari 2600 games using only the raw screen pixel inputs instead of a hand-coded representation of the game state. In a 2015 Nature paper, DeepMind presented a deep reinforcement learning system, termed deep Q-network (DQN), that was able to achieve superhuman performance on a number of Atari 2600 games using only the raw screen pixel inputs. What was particularly remarkable was how a single system (fixed input/output spaces, algorithm, and parameters), trained independently on each game, was able to perform well on such a large number of diverse games. More recently, OpenAI developed OpenAI Five, a team of five neural networks that can compete with amateur players in Dota 2.

The effectiveness of game engines & simulation platforms

It’s not just games that have played a central role in AI development. Game engines (and other simulation platforms) themselves are now becoming a powerful tool for researchers across many disciplines such as robotics, computer vision, autonomous vehicles, and natural language understanding.

A primary reason for adopting game engines for AI research is the ability to generate large amounts of synthetic data. This is exceptionally powerful as recent advances in AI and the availability of managed hardware in the cloud (e.g. GPUs, TPUs) have resulted in algorithms that can efficiently leverage huge volumes of data. Our partnership with DeepMind is one example of a premier research lab fully investing in utilizing virtual worlds to study AI. The use of game engines is even more profound in scenarios in which data set generation in the real world is prohibitively expensive or dangerous. A second reason for adopting game engines is their rendering quality and physics fidelity which enables the study of real-world problems in a safe and controlled environment. It also enables models trained on synthetic data to be transferred to the real world with minimal changes. A common example is training self-driving cars and Baidu’s move to leverage Unity to evaluate its algorithms is representative of an ongoing shift to embrace modern game engines.

AI is dubbed the new electricity due to its potential to transform multiple industries. We foresee game engines and simulation platforms playing a very important role in that transformation. This is evident by the large number of platforms that have recently been created to study a number of research problems such as playing video games, physics-based control, locomotion, 3D pose estimation, natural language instruction following, embodied question answering, and autonomous vehicles (e.g. Arcade Learning Environment, Starcraft II Learning Environment, ViZDoom, General Video Game AI, MuJoCo, Gibson, Allen Institute AI2-Thor, Facebook House3D, Microsoft AirSim, CARLA). The list also includes our own Unity ML-Agents Toolkit which can be used to transform any Unity scene into a learning environment to train intelligent agents using deep reinforcement learning and imitation learning algorithms. Consequently, we’re eager to encourage and foster AI research that leverages games and simulation platforms.

AAAI-19 workshop overview

At AAAI, later this month, we are co-organizing the Workshop in Games and Simulations for AI with Julian Togelius (Professor at New York University) and Roozbeh Mottaghi (Research Scientist at the Allen Institute for Artificial Intelligence). The workshop will include a full day of presentations by invited speakers and authors of peer-reviewed papers. The presentations will cover a number of topics including large-scale training of deep reinforcement learning systems such as AlphaGo, high-performance rendering for learning robot dexterity, learning to map natural language to controls of a quadcopter, and using drones to protect wildlife in the African savannah. If you are attending AAAI, join us at the workshop to learn more about how games and simulations are being used to power AI research.

Aloha!