Why use Colabs?

Writing a machine learning agent is hard.

If you only have access to a CPU, training a machine learning model can eat up days of your computers cycles just to get a single result.

Being human, we quickly lose interest in projects we are not able to interact with on a day to day basis. So, getting a single result every couple of days or even weeks(!) is simply an impractical way of doing research.

Also, on a personal note, it is even harder when you have a Macbook Air from 2011 & lack fancy things such as tons of RAM & a modern day GPU!!!

Enter Colaboratory

Late last year Google research released their machine learning tool, Colaboratory or Colabs for short.

Google Colaboratory.

The first release was pretty cool. It gave you access to a Jupyter notebook style front end which connected you to a virtual machine (VM) located on Google cloud — all for free.

However, what really peaked our interest back in late January (2018) was Google updated Colabs to give you free access to a GPU!!!

free,

access,

to,

a,

GPU,

Did we mention it was free? Holy mackerel balls.

(as a sidenote: Google is giving you free access to a VM with 11 GB of RAM & NVIDIA K80 GPU!!, last time we checked, NVIDIA K80 GPUs were going for $1500 on Amazon…)

NVIDIA Tesla K80 GPU

What then could be more amazing than training your machine learning model on the the cloud, completely for free, downloading your models’s learnings, and then seeing the results on YOUR computer in 12 hours as opposed to days or weeks.

How we did it.

The easy part

We immediately went to work trying to see if the headless Starcraft 2 version released last year (aka minus fancy Starcraft graphics) could download onto Colabs.

To our surprise it worked.

!wget http://blzdistsc2-a.akamaihd.net/Linux/SC2.4.0.2.zip

Resolving blzdistsc2-a.akamaihd.net (blzdistsc2-a.akamaihd.net)... 72.246.56.35, 72.246.56.72

Connecting to blzdistsc2-a.akamaihd.net (blzdistsc2-a.akamaihd.net)|72.246.56.35|:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: 3457493499 (3.2G) [application/zip]

Saving to: ‘SC2.4.0.2.zip’



SC2.4.0.2.zip 37%[======> ] 1.20G 28.2MB/s eta 84s --2018-04-10 03:11:41-- http://blzdistsc2-a.akamaihd.net/Linux/SC2.4.0.2.zip Resolving blzdistsc2-a.akamaihd.net (blzdistsc2-a.akamaihd.net)... 72.246.56.35, 72.246.56.72Connecting to blzdistsc2-a.akamaihd.net (blzdistsc2-a.akamaihd.net)|72.246.56.35|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 3457493499 (3.2G) [application/zip]Saving to: ‘SC2.4.0.2.zip’SC2.4.0.2.zip 37%[======> ] 1.20G 28.2MB/s eta 84s SC2.4.0.2.zip 100%[===================>] 3.22G 30.2MB/s in 2m 10s



2018-04-10 03:13:53 (25.3 MB/s) - ‘SC2.4.0.2.zip’ saved [3457493499/3457493499]

Next up on the agenda, was unzipping the headless client,

That worked too.

!unzip -P iagreetotheeula SC2.4.0.2.zip Archive: SC2.4.0.2.zip creating: StarCraftII/ creating: StarCraftII/Replays/ creating: StarCraftII/Battle.net/ creating: StarCraftII/Battle.net/Cache/ creating: StarCraftII/Battle.net/Cache/02/ creating: StarCraftII/Battle.net/Cache/02/4e/ inflating: StarCraftII/Battle.net/Cache/02/4e/024eaf5791a26ab0ab99e5c3ef5c2b96ebbdb176dd8862bfee38a9c3242e4115.s2ma

Of course you need Deepmind’s PySc2 in order to be able to able to get your Python code talking to Starcraft 2.

So, with a simple install command… viola! PySc2 is now installed on Colaboratory.

!pip install pysc2 Collecting pysc2 Downloading PySC2-1.2-py3-none-any.whl (112kB) 100% |████████████████████████████████| 112kB 2.1MB/s

The hard part

We then managed to fire up an instance of the headless client, but when PySc2 tried to connect via a technology known as websockets, we got a connection timeout error.

Well, that was a waste of time.

Not quite.

After an extreme amount of hard work, Frank He from our StarAi team managed to debug Colabs (without a debugger…) on the root cause of the error. For an excellent explanation of how he managed to do this, feel free to head on over to his medium post here.

It turned out there was a memory allocator error causing the SC2 headless client to crash on Colaboratory.

So now, we have a WORKING version of the headless client on Colabs talking to PySC2

Game on!

Training a Machine Learning Agent

Will’s DQN trained on Google Colabs.

In another recent success, one of our team members William Xu, managed to get a version of a machine learning algorithm known as DQN working on the “move to beacon” problem from the initial PySC2 release.

We decided to try Will’s agent on Colabs and to our never ending surprise, it started to train!!!

!PYTHONPATH=. python ~/pytorch/sc2_agents/BaseTrainer.py --map=MoveToBeacon --train=True Entering load game phase.

Launching next game.

Next launch phase started:



How to Train your agent.

First and most importantly, you will need a stable internet connection. The reason being that if you disconnect from Colabs whilst the model is training all your hard work is lost.

Ideally, it is best to leave your model training overnight so you can wake up the following morning to fresh machine learning results. If you are anything like us & your flatmates like to watch Netflix, perhaps try find a better internet connection where your model can train safely!

Also make sure that your computer does not time out to it’s lock screen. We have found that this seems to disconnect the internet too.

You will also need to optimise your machine learning model for Colabs, this step is easy & we have written a follow up article and in part 2 we show you how to optimize your model for Colabs

And that is it!!

Get started now >>

Below is a link to our Colabs notebook which, if you are using Google Chrome, you can load and play with in your browser right now!

Base agent

https://colab.research.google.com/drive/1AzCKV98UaQQz2aJIeGWlExcxBrpgKsIV

DQN Agent

https://drive.google.com/file/d/18asH_hmCjgUUCb-_-SPgPWkK0MYLPAmk/view?usp=sharing