As early as 1947 Alan Turing was thinking of chess as a good demonstration of machine intelligence, because it allowed for what he called “fair play for machines”. In a game like chess, the machine is interacting with the same pieces and board position according to the same rules as the human. Any strategy available to the human was in principle available to a clever enough machine. Turing experimented with his own chess-playing programs, but they were never very good. It was over forty years after Turing’s early death that IBM’s supercomputer Deep Blue, considering over 200 million moves a second, beat Garry Kasparov at chess. It took another 20 years of technological progress and theoretical innovations for AlphaGo to beat Ke Jie (and a bunch of other 9 dans) at go.

AI’ll do that to ya

These demonstrations are culturally and historically significant not only because they signal a technological revolution, but also because they represent machines “beating us at our own games”, so to speak. For thousands of years humanity has studied these games for lessons in deep thought and strategic insight. Chess and go have, to some extent, been a benchmark by which we’ve historically evaluated and ranked our own intelligence. So when the machines beat our top players at these games, to some extent it strikes a blow against humanity. We’ve selected the best representatives of our best games and they are defeated in legitimate battle by the mechanical upstarts. It doesn’t take mindless tech optimism to recognize these as epic, legendary events.

While StarCraft has been around for a fraction of the time of chess and go, in some ways it holds the same revered status in gaming communities today. StarCraft isn’t the most popular game from either a player or spectator perspective, but there’s a broad recognition across the big tent gaming community that top professional StarCraft play requires a degree of control, speed, and decision making that is intimidating even when compared to other esports. This is partly in virtue of StarCraft being one of the oldest and most professionalized games in the community.

Street Fighter has been around longer.

It is often remarked in these discussions that StarCraft is a game of imperfect information, since a fog of war hides the actions of your opponents outside your field of view. This is unlike chess and go, which are perfect information games where the opponent knows everything that is happening on the board. However, imperfect information doesn’t substantively change the “fairness” of the game, since vision works the same (and predictably) for each player, and can therefore be anticipated just like any other variable in the game. Nothing in StarCraft can be blamed on chance, despite imperfect information. Everything develops directly from the decisions and interactions of the opponents.

Given these considerations of fairness and the historic importance of human vs machine competition, the fact that AlphaStar operated on a full-map view is significant, in that it shatters the appearance of fair play. AlphaStar’s devs justified their approach by explaining that AlphaStar’s APM (actions per minute), SPM (screens per minute), and reaction times were actually lower than average professional play. Therefore, they argued, the advantage AlphaStar gains in the game can’t simply be attributed to AlphaStar’s speed. Instead, AlphaStar’s win should be attributed to its superior decision making at both the micro and macro level. But this argument stumbles in light of the live game after AlphaStar’s interface was changed to require more traditional screen management. MaNa easily defeated AlphaStar because AlphaStar was not making good unit decisions in the face of prism harass. The results suggest that AlphaStar’s advantage was not a product of good decision-making, and that DeepMind was overstating the strength of AlphaStar’s play.

I made these arguments in the comments of my Facebook thread during the broadcast last week:

Brownbear develops a similar argument in the video below.

Particularly interesting for me was how this dispute cast a new light on prior DeepMind victories. For instance, consider the famous Move 37 from game 2 of AlphaGo’s first public matches against Lee Sedol. In the clip below, the commentators initially suggest the move was a mistake, a “click-o” that in online play could justify a take-back, given the popular consensus that it was a bad move. But within the span of the clip, the commentators convince themselves that the move is actually very strong. AlphaGo eventually wins this game, and Move 37 has become mythologized as a symbol of how AlphaGo’s play is not only better than human play, but wildly counter to human intuitions about what good play looks like.

AlphaGo’s legendary move 37.

Similarly, the casters during AlphaStar’s games comment several times that the games are unlike those you’d see in professional play, especially strategies like the mass disruptors in the TLO games or mass stalkers in the games against MaNa. The devs at DeepMind repeatedly argue that AlphaStar’s strategy of over-saturating probes in the natural might be a lesson to professional players who tend to disfavor this strategy. The suggestion is that AlphaStar might be showing us superior play styles that top human players have overlooked or underestimated for whatever reasons.

This argument only works on the assumption that the game is played fairly, and that humans and machines has sufficiently equal footing within the game that strategies can be shared between us. If, in fact, either player is using techniques or abilities that are off-limits to the other for whatever reasons, the very capacity of the game to function as a fair measure of performance has been compromised. AlphaStar can afford to build an all-stalker composition against double robo immortals because it can blink its stalkers perfectly. It can afford to overmake probes or disruptors because it has enough unit control to compensate for whatever risks these mistakes engender. The machine is evaluating the risks of the game differently from a human because it is fundamentally interfacing with the game in a different way that the human.