Even in an Oculus Rift experience where players only move a couple of inches, the internal sensors or external tracking camera can make subtle mistakes. So your in-game perspective can slowly drift out of sync (no pun intended) — you might find yourself twisting your head to look straight forward in a game. Traveling longer distances amplifies this effect. Constantly changing direction amplifies it even more. While most games can pop the camera back in place without much trouble, reorienting the entire world around a car might be the ultimate form of distracted driving. And many of these problems didn’t come to light until Powers started actually test-driving the system. "Three weeks out, we were like — it's done, it's good, we'll just hang out till the shoot date," says Snyder. "Then we show up for the first testing, and we're like, 'Oh my God. Nothing's working.'"

In order to translate Powers' driving into virtual motion, the team had to tweak a physics engine that — in other games — is basically meant to make amateur drivers feel awesome. Cars move faster, change direction at the tap of a button, and treat the conservation of momentum as more of a loose guideline than a law. "When you play Grand Theft Auto or anything like that, you can basically flip the car back and forth and it'll just do whatever the heck you want," says Snyder. "The PhysX engine's momentum is just awful." That’s an especially large problem for drifting, says Ben Conrad, who directed the Virtual Drift video. "It breaks physics. If we were doing laps, I think we would have been done. And then Matt gets into the car and starts doing countersteering."

The computer in Powers’ car tracks which direction he’s turning, how fast he’s going, and what kind of inertia he’s probably experiencing as he races across the lot. Existing location sensors aren’t fast or accurate enough to correctly plot where he is, so all this other data is used to estimate where he’s probably headed in the game. But unless those calculations are perfect, what’s happening on the course only matches the game for short periods of time.

"I love gaming, and I hate the idea of sitting in my room for months playing the same game," says Glenn Snyder, one of two designers that oil company Castrol hired to create its Rift drifting sim. "I really want to go out in the real world and just modify the visuals a little bit, you know? Just see something different." Along with fellow technologist Adam Amaral, he essentially created a video game tuned to mimic real life, hacking a Roush Stage 3 Mustang into a giant controller. Virtual Drift, granted, does more than change the world a little bit. Instead of a parking lot, pro driver Matt Powers sees what looks like a giant ocean made of coal, where waves and crumbling cliffs indicate that he should turn or slide. But the basic principle holds.

So Powers ventured into a world that got further from reality by the second, just to measure how bad it could get. "I'd listen to them — 'No, don't turn yet, you're fine, you're good, okay now turn.' But in the virtual world, I'm way off course," he says. "The turn is still way up there, or I'd passed it." Eventually, Amaral and Snyder matched the virtual and real controls well enough to last for a few minutes at a time. Between laps, Powers drove the car to marked GPS coordinates, so the team could figure out where he was and reset everything. "We were just lining up landmarks so they can place me exactly in the virtual world where I am in the actual world," he says.

It's more of a location extrapolator than a location tracker

Unfortunately, it can be hard to see all this work — and its results — in either Conrad’s YouTube short or the immersive Gear VR version. The video jumps quickly between an in-game view, a camera in the front seat of the car, and more traditional racing shots from outside. It’s pleasantly dizzying, but it doesn’t fully take advantage of VR’s greatest strengths — like its ability to make someone feel like they’re really in something that’s moving simultaneously through two worlds. The real draw is just the fact that Powers did it at all.

Like so many other virtual reality experiences, Virtual Drift is unabashedly an advertisement. The technical pieces are largely hacked together, and it’s really more of a location extrapolator than a location tracker — in more concrete terms, imagine a VR system that guesses how the world should move based on how fast you’re walking, not where your head actually is. And companies like Google and Microsoft are already doing similar but far more advanced work in fields that range from augmented reality to robotics. Would the system, for example, be any good in Google’s self-driving cars? "I think they're a little ahead of us," says Snyder dryly. "Just a bit. They're pretty good."

Creating a cheap system that can make VR work over large distances at high speeds, though, is more than just a stunt. Even small-scale positional tracking took a long time to get right. Valve, for example, spent a long time working with QR code wallpaper before switching to the laser boxes it uses now. "I feel like Valve has it down. Their laser tracking is on point," says Snyder. "I'm really big on taking [VR] out of the world where it's small enough to use laser tracking, and bringing it out to just anywhere. Throw it in the back of a car, throw it on a skateboard or bike or put it in a backpack."

Very few people are willing to temporarily blind themselves on the road to see a different reality. But it’s symbolic of the weird, anything-goes place in which VR still exists. If things go well, maybe we’ll all drive with heads-up displays in 10 years. They’ll just rewrite the world instead of covering it up.… and they probably won’t fill everything with burning fields of coal.