Despite virtual reality’s recent renaissance, the technology still has some obvious problems. One, you look like a dumbass using it. Two, the stomach-churning mismatch between what you see and what you feel contributes to “virtual reality sickness." But there’s another, less obvious flaw that could add to that off-kilter sensation: an eye-focusing problem called vergence-accommodation conflict. It’s only less obvious because, well, you rarely experience it outside of virtual reality.

At SIGGRAPH in Los Angeles this week, Stanford professor Gordon Wetzstein and his colleagues are presenting a new head-mounted display that minimizes the vergence-accommodation conflict. This isn’t just some esoteric academic problem. Leading VR companies like Oculus and Microsoft know all too well their headsets are off, and Magic Leap, the super secret augmented reality company in Florida, is betting the house on finding a solution first. “It’s an exciting area of research,” says Martin Banks, a vision scientist at the University of California, Berkeley. “I think it’s going to be the next big thing in displays.”

https://www.youtube.com/watch?t=166&v=YJdMPUF8cDM

Okay okay, so what’s the big deal with the vergence-accommodation conflict? Two things happen when you simply “look” at an object. First, you point your eyeballs. If an object is close, your eyes naturally converge on it; if it’s far, they diverge. Hence, vergence. If your eyes don’t line up correctly, you end up seeing double.

The second thing that happens is the lenses inside your eyes focus on the object, aka accommodation. Normally, vergence and accommodation are coupled. “The visual system has developed a circuit where the two response talk to each other,” says Banks. “That makes perfect sense in the natural environment. They’re both trying to get to the same distance, so why wouldn’t they talk to one another?” In other words, your meat brain has figured out a handy shortcut for the real world.

Strap on an Oculus Rift or Samsung Gear VR, though, and all bets are off. These stereoscopic headsets create 3D images by showing the left and right eye images that are slightly offset—the more offset, the closer an object appears. That means your eyes are always accommodating to the screen strapped on your face, but they’re converging to a distance further off. Your eyes aren’t dumb; it’s totally possible to uncouple accommodation and convergence, but they won’t be happy about it. And they’ll be downright rebelling after a few hours.

Hoffman et al. Journal of Vision 2008

In 2008, Banks and his colleagues published a paper showing that the vergence-accommodation conflict causes fatigue and discomfort. It may also contribute to virtual reality sickness. To do the research, the team had to build a device that basically solves the conflict. The device is big box—like you might see at an optometrist's office—with three or four different planes each showing a different part of an image. A lens that can change power sits in front of the planes, bringing different parts of the image into focus. It has great color and great resolution, but it’s not coming to your living room any time soon. “We actually put people in a bite bar,” says Banks. “We have to know exactly where the eye is, so you can’t move your head.”

The head-mounted display from Wetzstein and colleagues at SIGGRAPH, on the other hand, actually fits on your head. Your eyes can move freely. Instead of flat images, the display creates light fields, or 3D patterns of light rays that mimic light bouncing off objects in the real world. The hardware consists of two stacked liquid crystal displays (LCD). An algorithm divides an image between the two LCDs, with more detail for distant objects on the rear LCD. “Where the real difference comes in is that computational step,” says Wetzstein. The entire device is made with off-the-shelf parts.

Wetzstein et al 2015

But again, don’t expect to play video games anytime soon in this display either. The field of view is limited at 90 degrees, and resolution isn’t up to Oculus standards. More planes could mean more realistic images. But it does show that a truly head-mounted display can deal with the vergence-accommodation conflict. Other groups have tried to create light fields by bending light with microlenses, but getting high resolution on those is even more difficult, says Wetzstein. Magic Leap is also doing something with light fields, but nobody's sure exactly what.

The current crop of high-profile head-mounted displays from Oculus and Samsung rely on stereoscopic displays. They’re pretty good, but they’re likely a dead end for solving the vergence-accommodation conflict. To do that, the industry might have to start over with light fields.