I spent a good couple of weekends last month playing alien-busting strategy sequel XCOM 2. I enjoyed a lot of my time with the game, but it also frustrated me a bunch of times as well – in particular, it frustrated me in a lot of ways that Invisible Inc, Klei’s 2015 sneaky masterpiece, didn’t. After completing my XCOM 2 campaign and going back to Invisible Inc for a bit of mulling, I think I’ve come to some conclusions about an important way the two games differ, and how it reveals subtle problems with how procedural generation interacts with other game systems. I want to tell you why I think Invisible Inc structures itself better around procedural content and why I think it’s important (and why my opinion might not matter, too).

XCOM 2 and Invisible Inc both use generated content to provide a dynamic and shifting set of challenges for the player. Their strategic level is generated, with different missions made available to choose from. The missions themselves have generated layouts with procedural arrangements of enemies, cover, spaces, loot and objectives. There are also opportunities to upgrade your squads which are partly dependent on random elements – only certain items might be available at any one time in Invisible Inc, while XCOM 2 randomises the results of some kinds of research and restricts expansion through randomised resource dumps. On paper, they follow very similar patterns, and then add a layer a fixed set of tactical interactions so the player is always solving the same kinds of problem, but usually in completely new or previously unseen configurations.

I think it’s important to distinguish between ‘random generation’, which is a term we sometimes use incorrectly, from ‘procedural generation’. The former implies a certain kind of blindness that usually isn’t actually true in games. When a rookie levels up in XCOM 2 it’s probably a fair dice roll to decide what class they become. But a lot of generative processes in games follow very strict rules instead of dice rolls – that’s why we call them procedural (they follow a procedure). Invisible Inc never places cameras in the starting room, for example, and XCOM 2 always puts a squad of aliens covering black site objectives. Even the combat dice rolls in XCOM 2 aren’t entirely random – they’re juiced to suit the situation, reducing hit chances for aliens if they get too lucky.

On the face of it, both games have the same unpredictability, and the same potential to cause frustration. Invisible Inc has generated me levels with impassable choke points and perfectly overlapping guard patrols. XCOM 2 has overlapped alien aggro zones in front of wide open death corridors with no cover. It’s very hard – extremely hard, in fact – to stop this from happening if you develop procedural generators in the usual way. When developers write procedural generators, they usually test them by looking at the output. You write an algorithm, imagine how the output will look, test it a bit, and if it looks roughly okay then it’s good to test. The problem is that a level generator might be able to produce millions of different levels, and manual testing can only show you a dozen, or a hundred, or even a thousand. It’s really hard to know if that thousand levels is a good sample, or if they’re all clustered around some average level. What are the extremes of your generator? What are the worst-case (or best-case) scenarios it can output? You don’t know.

You can mitigate this a bit. You can do lots of playtesting, which means you get a lot more people testing your generator, and if something happens only 10% of the time you’ll probably find out. But it’s still just more samples, there’s no way of really knowing whether your procedural generator will behave in that weird 1% or 0.1% of the time. In any case, some people don’t want it to behave – they like the wild, untamed edges of the generative space. Whether you want it or not, though, once you release your game people are going to encounter the unusual, unpredictable, edge cases of your generator. And this is the point at which XCOM 2 and Invisible Inc completely diverge, in my opinion.

Invisible Inc’s level generator is really fickle. Sometimes I’ll leave an innocuous corner unexplored only to discover it housed the exit twenty turns later. Other times, I’ll find the exit in the first room I enter, and the objective next door. In terms of variability, Invisible Inc’s generator has way more strange extremes and edge cases than XCOM 2’s generator, which is generally more consistent. But it frustrates me a lot less because a fundamental principle of Invisible Inc is that the player always acts with total precision. This means that I can make a decision about how I want to solve this new problem the generator has presented me, and know exactly how it will play out.

What I mean by this is that almost every single interaction in Invisible Inc, with a few small exceptions, is deterministic. Attacks never miss, hacking always succeeds, vision is always explicit and consistent. Some effects are randomised – rewinding a turn where a guard spawned can cause it to spawn at a different location, for example. But in general, when I am planning my turn, I know what the effects of my actions will be. XCOM 2 builds itself a different way – its combat is famously built around percentage chances to hit, it hides knowledge from the player, and most problematically it also unintentionally hides knowledge that the player is supposed to have, through patchy UI design.

Why is this such a big deal? Because sooner or later, your content generator is going to make a mistake, and it’s going to make the player feel like you’re not playing fairly. The role of a level generator in a strategy game like this is almost like a DM in a pen and paper game: it’s setting up a scenario, and asking if you can solve it, and the implication here is that a solution exists because the DM is really there to make sure you’re having a good time. When a generator messes up, it feels like you’re being treated like a fool – the DM is either stupid or vindictive, throwing down eight dragons and smirking at you when you protest.

In XCOM 2, it’s possible to encounter a hiccup in the game’s generated content, and respond to it in a completely sensible and strategically clever way – but if the dice roll against you, you will fail. To some people this is simply the appeal of the XCOM series, but I doubt that many other games could get away with treating the player like this, particularly games with such a fierce potential for death spiralling. XCOM 2 seems to hope that everything sort of comes out in the wash – that the troughs of randomness are evened out by peaks of good luck. For a lot of players this is exactly what will happen. But I’m much more interested in the extreme cases – the people who either encountered trivially easy or totally impossible content, and never encountered the idealised average game of XCOM 2 that the designers intended for.

For the record, I’m not saying that XCOM 2 is bad. A friend pointed out that changing XCOM’s combat system and removing all dice rolls might be good or bad, but it definitely wouldn’t be XCOM. He’s totally right – that game is everything it aspires to be. But I think it shows why it’s important to provide the player with the tools to overcome the unpredictable systems we put in our games, an emergency toolkit that gives them a way to reassert control over the chaos of procedural content. Even Spelunky, a game with one of the most consistent and conservatively-designed procedural generators ever, gave the player tricks to use in the event of bad level generation. Whatever happens, whatever scrape the player gets into, they’re given four bombs to blow through walls and floors with. Spelunky doesn’t hope for averages – it plans for the worst, and gives players fun tools to get around problems when they arise.

Rogue Process uses some of these ideas to help the player break through blips in our generated skyscrapers, but I’m also hoping to build analysers into the game itself to help make sure truly bad scenarios are fewer and further between. I’ll talk about analysing procedural generators and understanding their output in another blog post, and why I think it’s important for some games to be more restrictive in how they generate content. I might even have a little interactive generator demo by then for you to play with, too. In the meantime – thanks for reading! Let me know what you think about the two games in the comments or say hi on Twitter.