When Facebook bought Oculus VR back in March of 2014, many wondered exactly what the social network was going to do with it—let's face it, many of us are still wondering. But there are some interesting bits of tech starting to emerge from the now Facebook-owned Oculus that hint at what the future might hold for the Rift outside gaming. One such piece of tech—a "facial performance" tracking system—adds a vital element of social interaction to VR usage: facial expressions.

Researchers at University of Southern California (with help from Facebook) have devised a system that tracks a user's facial expressions and translates them onto an avatar in the VR world. It works by using an off-the-shelf Intel RealSense 3D Camera bolted to the front of an Oculus Rift DK2 in order to capture facial movements for the lower half of the face. The really clever part, though, is how it captures movements for the top half of the face, which is obviously covered up.

The researchers mounted eight strain gauges inside the foam liner of the Rift and developed software based on the Facial Action Coding System (FACS) often used by animators to integrate the data from the depth-sensing camera, strain gauges, and the Rift itself. The result is an eerily accurate representation of the user's facial expressions, down to the smallest of movements. Even better, latency was generally low, with the researchers measuring 3ms for facial feature detection, 5ms for blend shape optimisation, and 3ms for the mapping in software.

The applications for the social interactions in the VR space using the technology are obvious: users would be able to take part in real-time meets via an avatar, yet still convey important emotions in conversation. There's definitely scope for the using the technology in games too: imagine a VR RPG where, instead of just creating a character and selecting from pre-baked conversational options during gameplay, virtual characters directly could react to your facial expressions and mood in order to generate different dialogue.

While the tech is interesting, naturally there's a catch—and it's not just that this is a research project. Currently, in order for the system to work correctly, it needs to calibrate to a user's face without the front of the headset attached. Only after this initial calibration step (which involves pulling lots of silly faces), is the system able to accurately measure the user's face. Unsurprisingly, it currently needs a rather powerful rig to run well: powered by a Core i7-4820K, 32GB of RAM, and a GTX 980, the system renders at a steady 30 FPS.

These aren't insurmountable problems by any means, and the researchers reckon they'll be able to remove that initial calibration step by feeding the software more data on different faces. They also think it wouldn't be all that hard to design a smaller camera and integrate it into the bottom of the headset to create a more commercially viable product.

That's probably some ways off at this point, though, with Oculus having finally settled on a concrete design and specification for the Rift after years of research and development. The first consumer version of the Oculus Rift is set to launch early next year, but make sure your hardware is up to scratch if you want to be an early adopter: there are some hefty system requirements.