So it appears the Oculus Rift is really happening. A buddy of mine went in early on the kickstarter, and his will supposedly be in the mail some time this week. In a way the Oculus Rift, or, more precisely, the most recent foray of VR into the mainstream that it embodies, was the reason why I started this blog in the first place. I’m very much looking forward to it (more on that below), but I’m also somewhat worried that the huge level of pre-release excitement in the gaming world might turn into a backlash against VR in general. So I made a video laying out my opinions (see Figure 1, or the embedded video below).

Before I start worrying, here is why I’m excited: I already have two HMDs, an eMagin Z800 Visor and a Sony HMZ-T1 (in the video above, I’m using the Z800). Using the Vrui VR toolkit, these HMDs (and the Oculus Rift as well once I get my hands on one) are drop-in replacements for each other and for projection-based holographic displays such as CAVEs. This makes it easy to compare the same applications in different displays. And some applications that feel very immersive in a CAVE, such as the Quake ||| Arena map viewer, feel much less so with my HMDs. Concretely, I am slightly afraid of heights, and when I step up to a ledge in a Quake map, my hands get clammy and my feet start to tingle; when I jump down, I feel my stomach lifting. There’s nothing I can do about it; I’ve done this for eight years, and it happens every time. I recently tried the same thing with an HMD, and nothing happened. Why? Not sure. My HMDs have narrow fields of view (around 45°), and I can see the real room I’m in from the corners of my eyes. With the Oculus Rift and its wide FOV, I’ll be able to investigate in more detail, and I’m optimistic that immersion will be improved, hopefully to the level of the CAVE.

So on to the worrying. In detail, I’m concerned about two things: one, that audiences who have never been exposed to VR or HMDs have unrealistic expectations of what the Oculus Rift can do, and will be disappointed when they see the real thing; two, that software ported from the desktop world will carry over baggage from that world that will lead to a sub-optimal experience. There are also some really basic things HMD-based 3D software could get wrong, but I’m hoping that a good high-level SDK will take care of those.

These are the three core issues (addressed in the video):

The lack of positional head tracking in the Oculus Rift, at least out-of-the-box. Update: I have been corrected; the Oculus team plans to include some form of positional head tracking in the final consumer version that is planned to come out next year. That is very good news indeed; I’m waiting to hear more details. Controller-based navigation, as carried over from the desktop and its WASD paradigm. View-linked aiming, also carried over from the desktop.

The first in an expectation problem. When people hear about HMDs, they automatically assume it will support things like peeking around corners (at least the ones I’ve talked to), or looking at things from different angles, etc. Problem: without positional head tracking, that’s not possible. Now gamers might argue that that’s irrelevant — there are buttons on the controller for leaning etc. — but they’re missing an important fact: we move our heads subconsciously all the time. And if the virtual world presented in the HMD doesn’t react to that like the real world would, the illusion breaks down. As an aside, I actually know a good number of gamers that do “desktop gymnastics” when playing, so to speak. They lift out of their chair to peek over ledges, lean to look around corners, and all that without the game reacting in any way (I once had a roommate who broke a vase trying to dodge an incoming fireball in Doom). The world not reacting may be OK on a desktop (broken tchotchkes aside), but maybe not in an environment as convincing as what’s presented by an HMD — there it will lead to motion sickness. My advice: start thinking about aftermarket positional head trackers. For example, put some LED sockets in well-known positions onto the device, so that users can add their own LEDs and use a high-framerate camera like the Playstation Eye for DIY optical head tracking. Update: It appears my advice is too late; they’re already thinking about positional head trackers. More details TBD.

The second and third issues are user interface issues, and those need to be addressed by developers porting their games to HMDs. I’m not up-to-date on my demos, but what I’ve seen so far was not great, mostly with 1:1 ported desktop interfaces. The best thing (really the only good thing) I’ve seen so far is Project Holodeck, out of USC. They have position tracking, position- and orientation-tracked 3D input devices, i.e., all the good stuff that’s required to build a head-mounted VR environment that deserves the name. The fact that they’re directly talking to the people behind the Oculus Rift is very reassuring. I don’t know anything about Project Holodeck’s software stack, but I’m hoping that the VR nuts-and-bolts (virtual camera set-up, tracking, navigation, interaction, …) are not individually coded into each application, but instead provided by a shared high-level toolkit (I know I keep repeating myself).

So what’s to do? First, everybody involved needs to be a bit more grounded when describing the capabilities of HMDs in general, and the Oculus Rift in particular, to the world at large. The hype getting any more hype-y is not good for anyone. How to do that properly, for example, how to describe the admittedly subtle differences between levels of head tracking (none, orientation only, orientation+position) to a “lay” audience, I honestly don’t know. We’ll have to just keep trying.

Second, the VR community should reach out more to the game developer community. A lot of research has been done, and there’s no need to re-invent the wheel ten more times, or repeat old mistakes.

To wrap it up, and because this post ran long, I want to repeat my core message: I’m not concerned about the Oculus Rift display hardware at all. I’m very much looking forward to supporting it in the Vrui VR toolkit, and using it for my own research (and entertainment) in the future. I’m only worried about how it’s going to be used by games or other 3D software. What’s been holding VR back is not bad hardware, but bad software, and if the Oculus Rift ends up mostly exposing the world to bad VR software, we’re all in trouble.