Back in January, we saw DeepMind announce AlphaStar, a bot that used reinforcement learning to beat human players at StarCraft II. One of the key changes that AlphaStar leveraged was a new set of features that enabled the bot to observe the entire map at once, known as raw observations, and act anywhere on the map without having to move the screen first, know as raw actions.

Observing and acting in this way can open up a whole new set of possibilities while also presenting some new challenges. Most significantly any existing bots will not be able to immediately use the new functionality and will need to be trained from scratch. In this tutorial we’ll explore how to implement these new tools so that you can apply them to your own bot.

This tutorial assumes that you are familiar with PySC2 from my previous tutorial.

Note: Development on raw observations and actions is still ongoing, so some things may not work or may change. While this tutorial will work on any platform, in order to fully utilise raw actions you should use the Linux version of StarCraft II with the updated stableid.json file, otherwise there are some actions, such as move, that will not work.

1. Install the “dev” Branch of PySC2

The new features are only available on the “dev” branch of the PySC2 Git repository. First you should clone the PySC2 repository:

git clone https://github.com/deepmind/pysc2.git

Next, switch to the dev branch:

git checkout -t origin/dev

Now install PySC2 from this location:

pip install -e .

It might be necessary for you to install with --force-reinstall to make sure that all dependencies are updated.

2. Create the Basic Agent

First let’s add the imports:

Next we can create a really basic agent:

Already you are using raw actions! It might only be a no-op, but it’s still cool.

Now let’s add the code that runs everything:

The action space is set to RAW , this enables the raw actions instead of the regular screen and mini-map actions.

The observation space is set to raw which is much the same as use_feature_units except that it enables a few more properties for each unit, and the unit’s x and y coordinates are represented in world space.

We convert the raw resolution from world space to 64, the same as the default mini-map resolution in feature space, which is handy if you are transitioning old code.

If you run this code now, it will do nothing and your agent will eventually lose.

python raw_agent.py

The code for this step is available here.

3. Determine the Base Location

Before we can build a Zealot, we need to build a Gateway, which needs a Pylon. Before we can build a Pylon we need to know if our base is at the top-left, or if it’s at the bottom right. Let’s add a property to track this:

Next we can add a utility method to help us to find raw units we own, with a given type:

Note how we are using obs.observation.raw_units here just like we have used obs.observation.feature_units in previous tutorials.

By default the raw units include enemy and neutral units, such as mineral patches, so we filter by unit.alliance to make sure the units belong to us, and by unit.unit_type to make sure we only get units of a specific type.

Now in the step() method lets determine the base position based on the location of our Nexus:

Since the map coordinates are restricted to 64x64 and we are using the Simple64 map, any x coordinate from 0–31 means the Nexus is on the left, and since the only spawn location on the left is at the top, we can assume the base is at top left. Any value from 32–63 would mean that the base is at bottom-right.

The code for this step is available here.

4. Build a Pylon

Great, we know where our base is located, so let’s use a probe to build a Pylon. The awesome thing about raw actions is we don’t have to select units, we can simply instruct a unit to perform an action by using the unit’s tag. The tag is a unique identifier for that instance of the unit within the game, so every unit will have a different identifier.

Since we only want to build a Pylon if one doesn’t exist, let’s check if we have an existing Pylon and if we have enough minerals to build one:

Now let’s get all of our Probes:

Note how we check if there are any Probes — this will prevent a crash in case the enemy has killed all of the Probes and there are none to use.

Next we choose the Pylon build location in the world space (scaled to 64x64) depending on the base location:

Now for simplicity we want to use the Probe that is closest to the Pylon build location, we do this by calculating the distance from each Probe to the build location. For this we need to add a new method to the class:

Then we call this method with our list of Probes and the build location:

We then extract the Probe with the shortest distance:

Finally, we tell the Probe to build the Pylon:

This last line will look fairly familiar if you have used regular actions before, however there are some slight differences. As you can see we are using RAW_FUNCTIONS which generally come in “point”, “unit”, “autocast” and “quick” formats.

The first parameter determines if the command is performed immediately, or if it is queued with existing commands. The second parameter is the unit’s tag. The third parameter is a list or tuple containing the x and y coordinates.

One thing to note is that you don’t need to check if the action is available, your action will simply be ignored if it is not possible.

If you run the code your probe should build a Pylon.

The code for this step is available here.

5. Build a Gateway

Now that we have a Pylon we can build a Gateway, the code is very similar. First we want to check for completed Pylons so we need to add another utility method:

Then we can use this method to get the list of completed Pylons:

Then we need to get any existing Gateways:

Next we want to check everything and get our Probes:

Next we determine the Gateway build location:

Then we can calculate the distances again, find the closest Probe (which should be the same one) and tell it to build the Gateway:

Run the code and watch the glorious Gateway construction!

The code for this step is available here.

6. Train Some Zealots

Awesome, we have a Pylon, we have a Gateway, now all we need is an army.

Let’s get the completed gateways:

Next we can calculate free supply:

Before we can build a Zealot, we need to make sure we have everything we need:

Next, get the Gateway:

One cool feature of raw units is we can see the number of “orders” that each unit has, in this case the order length represents the build queue of the Gateway. Since the Gateway can only train five Zealots at a time, we can check before issuing the action:

When you run the code your Gateway will train four Zealots who will defend your base vigorously if the enemy approaches.

The code for this step is available here.

7. Attack!

Cool, we have Zealots, now we need to attack.

Similar to before, we can get all of our Zealots:

Now we only want to attack once we have amassed a small army, essentially once we have exhausted our free supply:

Next we choose the attack coordinates depending on our base location:

At this point you would normally select the army right? Well you don’t need to select the army, but you do have the tags for all Zealots. Ideally you could create a list of Zealot tags for use by the action, however there is currently a bug in PySC2 that only lets you control one unit at a time. Sad face.

While my pull request is waiting to be approved we will try an alternative approach. First we calculate the distances from each Zealot to the attack location:

Then we choose the Zealot that is furthest:

Note that we use np.argmax here to select the furthest instead of np.argmin that we use earlier to find the closest.

Now we want the Zealots to search around the enemy base location a little bit, so we randomly choose a location within 4 points of the attack_xy and order the Zealot to attack:

When you run your code, your Zealots will attack and will win almost every time.

The completed code is available here.

If you like this tutorial, please support me on Patreon. Also please join me on Discord, or follow me on Twitch, Medium, GitHub, Twitter and YouTube.