As you may or may not know, Google Stadia is based on Linux. So for games to be able to run on Stadia, they will have to be ported to Linux – or – written for Linux from scratch. As one game developer has now revealed, there is a bug in the Linux kernel’s scheduler which can cause massive latency issues.

Everything at Google runs on Linux. Search, Gmail, Android, you name it. When you access any Google service over the Internet, your computer or smartphone is connecting to Linux-based servers hosted on Google’s Cloud Platform (GCP). And GCP, in turn, is the basis for Stadia.

Google’s cloud gaming service has had a pretty rough launch, to put it lightly. Stadia has been marred by pre-order issues, an insufficient catalog of launch titles, missing features, bad reviews, and laggy games. Lag, which is often caused by latency, has always been the main concern for most people when it comes to streaming games over the Internet. And unfortunately, those concerns weren’t exactly unfounded.

Game developer uncovers Linux bug

As it turns out now, one of the reasons for Stadia’s latency issues lies in the Linux kernel’s scheduler. Malte Skarupke, a game developer and AI programmer at Avalanche Studios (Rage 2) has brought the issue forward on his blog:

“So this all started like this: I overheard somebody at work complaining about mysterious stalls while porting Rage 2 to Stadia. The only thing those mysterious stalls had in common was that they were all using spinlocks. I was curious about that because I happened to be the person who wrote the spinlock we were using. The problem was that there was a thread that spent several milliseconds trying to acquire a spinlock at a time when no other thread was holding the spinlock. Let me repeat that: The spinlock was free to take yet a thread took multiple milliseconds to acquire it. In a video game, where you have to get a picture on the screen every 16 ms or 33 ms (depending on if you’re running at 60hz or 30hz) a stall that takes more than a millisecond is terrible. Especially if you’re literally stalling all threads. (as was happening here) In our case we were able to make the problem go away by replacing spinlocks with mutexes […]”

He goes on in great length, describing how the problem is caused by the Linux scheduler and how to possibly work around it. For those of you interested in the nitty-gritty technical details, head on over to his blog post. A highly recommended read.

Malte later responded to a comment made on his post by Con Kolivas, lead developer of MuQSS – an alternative scheduler for Linux:

I know that we were not the only developers who had problems with the scheduler on Stadia. And Google is very aware of the problem. They care a lot about latency because latency is super important for the Stadia experience. And one of the ways they’ve reduced latency is to run games at 60hz that run at 30hz on console. But that means you only have 16ms to get a frame on the screen, and if the scheduler gives you a random hitch of a millisecond, you’re screwed. Malte Skarupke

So there you have it. Stadia’s latency issues probably have a multitude of different reasons – network latency being the most obvious – and this bug in Linux is apparently one of them.