We've introduced our Age of Ascent coverage yesterday, by announcing the Xbox One port of the ultra-MMO. Today, let's take a closer look at how Illyriad Games is using Azure's cloud to make 50K in the same battle a reality, thanks to CEO James Niesewand.

How exactly are you leveraging the power of Microsoft Azure's cloud, and why did you choose this particular service over others?

We knew that the only possible solution to the sheer network and compute problem thrown up by AoA's scale could only be handled in the cloud, as any attempt to provide your own inhouse solution would be both prohibitively expensive and utter madness.

The load has to be spread out. And not only that, it has to spread out and to fold and unfold space based on unanticipated demand. Say, for example, 2 large player clans suddenly decide to go to war on a whim (over asteroid mining rights or, more likely, some smacktalk in chat!) and they suddenly meet up in an area of space to duke it out. As the number of players engaged in combat in that small chunk of the universe skyrockets from 50 players to 500 players to 5000 players to 50k players... we have to be able to cope with the load seamlessly.

And that means bringing in additional servers rapidly on demand, quickly bringing the new nodes up to speed with what's happening in realtime and then sharing the fluctuating load amongst them - without skipping a beat. After the spacedust has settled on the debris field we then have to de-provision those additional servers so that we're not (as a company) paying for server resource that we no longer need.

We chose Microsoft Azure over other cloud solutions for 3 main reasons:

Firstly, we're extremely familiar with Microsoft technologies; we know what they're awesome at and we know where the rough edges are - so it's a natural fit that way.

Secondly, Microsoft themselves have been extremely supportive, providing assistance and advice throughout. Uniquely amongst all the people we talked to about what we were trying to do with AoA their first reaction wasn't a wave of laughter 🙂 but instead the offer of an Architectural Design Session to validate and critique our plans. That relationship has continued and deepened throughout the development process and has given us access to build AoA against, and to influence the direction of, essential core Azure technologies such as Service Fabric and Nanoserver. The BizSpark programme, where MS give free Azure time and software licences to startups, has also been exceptionally useful.

Thirdly, Microsoft's decision to open-source .NET and ASP.NET a couple of years back made it a no-brainer. We're extremely focused on open-source and open-web and we participate in W3C activities, contribute conformance tests to the Khronos group, and work exclusively at the front-end in HTML5 technologies - which is why we insist that AoA needs to not only run natively without plugins or installs in the web browser, but that it should also run without requiring expensive hardware. AoA runs happily at 30fps on a $200 chromebook, for example.

When MS open-sourced we could stop working on our own server solutions and start working, in conjunction with Microsoft and the rest of the open-source community, on solutions that would benefit everyone.

For example, the new MS libuv-based webserver called Kestrel that's currently in development for ASP.NET 5 Core is open source. Ben, our CTO, is the #1 contributor to this project, including Microsoft developers, and has helped make it more than 10 times faster than the previous version, so that it's now pushing way over 1.25m requests per second. We didn't do this solely out of the goodness of our hearts, but because our game actually needs our Azure webservers to be handle millions of http requests per second.

So it's true, then, that the Azure cloud can actually provide additional computing power like Microsoft advertised for its Xbox One console? Most people dismissed that as a joke.



Yes, it is absolutely true.

Games like Titanfall on the Xbox used Azure to provide locality, dedicated servers and an external AI system, and similarly Crackdown3 uses the compute power of Azure to make entire cities destructible. However, both games have their gameplay focus on small-team multiplayer rather than massive/ultra concurrency amongst the players themselves - which is a very different set of challenges to overcome.

The gameplay model for AoA has a high-concurrency multiplayer focus from the ground up, where we treat the game client essentially as an input/output and rendering device; and use Azure to spread out the staggeringly-large (and rapidly fluctuating) bandwidth and compute demands that we have to cope with.

But yes, the essential purpose of the cloud is to offload compute and I/O, and to scale them rapidly, as required. One of the main challenges is getting the results of that cloud-distributed compute back to the clients that need to know, whilst keeping track of the bigger picture across all of the cloud nodes and clusters that need to know more.

Why do you think AAA games haven't really used the cloud until Crackdown? Do you reckon there will be an increase in the usage of this technology even in big projects?

Fundamentally, I think it's because the technology is both new and "difficult". I don't mean, necessarily, that it's hard to use. I do, however, mean that it requires a very different mindset to your typical programming model. You have to abandon everything you think you know about typical client-server architectures, and think very 'differently' about what you're trying to accomplish with a cloud architecture. Writing for the cloud is entirely different, and requires different modes of thought.

For example, when I was speaking at //build a couple of months ago, I was asked what our "server tick" was. (For those who don't know, you can consider the server 'tick' as the fundamental "framerate" of the game's MMO engine).

And the question flummoxed me: because we don't have a server "tick". We sorta have an average physics/IM framerate across the cluster(s); but the nanosecond that a server is getting too hot, you pull another server in and 'share' the heat. In the cloud, you want your servers to work as hard as they can happily work - no more, no less - and you pull in more resource as the demand increases. There's not really any kind of 'heartbeat' to a cloud system and there shouldn't be - by design!

I also think that a lot of the game engine tools that many game developers use are horribly ill-equipped for cloud infrastructures, and that's certainly a limiting factor as well. Writing a cloud-savvy game does *not* mean running the same server hardware in the cloud, rather than in your own datacenter 🙂