This blog will start with a few entries on basic VR concepts as our R&D team tackles the question of streaming. There are a few excellent streaming solutions on the market right now including ALVR, VirtualDesktop and Riftcat. However no single solution in perfect. Reddit and Facebook are full of questions about streaming, what GPU to use, if RTX helps, why motion sickness occurs, what Bitrate is best, how to set up 5Ghz wifi and much more. In the following weeks and days, we aim to answer all of those questions while we research the tech.

Simulation Sickness

One of the major issues developers and players face when dealing with VR is motion sickness – usually called “simulation sickness” when dealing with VR. Simulation sickness occurs for a number of reasons. Let’s look at them one by one.

Lack of Movement

When you move in real life, your body perceives that movement in a number of ways. The most important ones are what you see, what your sense of balance is telling you and what your sense of motion is telling you. The latter two are mostly handled by your inner ear.

Now, if you play a game such as Beat Saber or Nympo Trainer where the majority of your character movements are determined by your own movements in roomscale, this works great since all senses align. However if you play games like Subnautica or Amoreon where your character moves fluidly in the room while you stand/sit still, you have a conflict between your senses. Your eyes say you’re moving while your inner ear says you’re standing still. This causes nausea in most people.

Lag

I feel stupid writing this, but real life doesn’t lag. No part of our body is built to handle lag. If you play a game traditionally, how much lag is tolerable largely depends on the kind of game you are playing. In a competitive shooter, anything above a few milliseconds is unacceptable. In a cinematic game like Assassin’s Creed, even half a second isn’t the end of the world. However in VR, almost no lag is acceptable. If you turn your head and there is just no world where you are looking to – or worse, if the virtual turn speed doesn’t match your physical turn speed – your brain freaks out. That is why most VR games are optimized to deliver exactly one frame every 7 milliseconds come what may. This also creates a massive issue when streaming.

Stream Lag

So if traditional VR does everything it can to deliver a frame every 7ms, what happens when you stream a game to your headset? Well, under ideal circumstances with a very powerful gaming PC, a dedicated 5Ghz hotspot right next to the headset and an overclocked Oculus Quest, we were able to get lag down to around 50ms. That’s literally 7 times the lag of a wired headset. We will look into the components of that lag in a later post but the message is clear: For any intense game, streaming right now just isn’t an option. That said, it is surprisingly usable for more relaxed and cinematic games.

Lag Inconcistency

The last element we will look at is inconsistent lag. Let’s say you are streaming your game to your headset and something causes interference. This is very common. Your neighbor’s microwave or a smartphone trying to connect to your network are enough. Suddenly your lag goes up. Then it goes down again. In real world circumstances this happens all the time. Within a single second, your lag – even under ideal circumstances – is bound to fluctuate between 50ms and 85ms. That may not sound like much, but it is critical for fluid movement.

Games where you warp to different locations are mostly unaffected by lag inconsistency. Your brain is unlikely to notice. But games where you fluidly move to a new location are impacted greatly. This is because your brain constantly makes calculations based on what it perceives around it. It uses the speed it sees objects move past you to estimate the speed you are moving at and so on. So now what happens if frames don’t come in at a set 50ms delay but have up to 30ms of spread between them? Well, ultimately the speed you brain perceives changes by up to 60 percent. Roughly 70 times a second.

This is insanely confusing to your brain – while you won’t even notice it on a conscious level. In addition to the sensory miss-match between your eye and your ear, you now also have a sensory mismatch between different data points your eye delivers. If you aren’t used to it, this will make you extremely motion sick. In our testing, even experienced VR players were only able to do minutes at most when playing games like Subnautica with current streaming technology.

TL;DR

Streaming introduces inconsistent lag to VR. Lag – among other factors – causes motion sickness. The larger and more inconsistent it is, the worse. Games where you move in room-scale and that are cinematic are much less affected than games that use fluid movements. Future blog posts will go into the technologies used to minimize this issue.