“We wanted to create a system that lets you move and explore a VR world just as naturally and easily as you would in real life,” says Kozminski.

Kozminski joined a team whose mission was to create the first full-featured “inside-out” tracking system for a consumer VR device. The technology would have to track the full range of a person’s movements (known as six degrees of freedom) and be able to pinpoint the location of the two handheld controllers as well as the headset.

Previously, VR devices relied on external sensors to track these movements. These cameras attach to a PC, and while they work well, they make VR less portable and more complicated to set up.

“With inside-out tracking in the headset, VR becomes as easy as putting on headphones to listen to music,” says Kozminski.

But the team’s mission was far from easy. They had to take state-of-the-art computer vision technology from the research lab and make it work in a consumer device that anyone could use. The tracking had to be accurate down to less than a millimeter — enough to capture a subtle tilt of your head or twitch of your hand. It had to be robust enough to work in a nearly infinite variety of conditions found in real-world homes. And it had to be efficient enough to work on a battery-powered device like Oculus Quest.