OSVR 0.6 Released!

This release of the OSVR projects brings dozens of updates for OSVR, including many improvements submitted by the community of OSVR developers on GitHub. Thanks to all of the contributors to OSVR!

Below is a short-form description of the major additions to OSVR. We just have too many new things to fit in one email. Visit this page for a full description of the many changes and additions in OSVR as well as a list of "contributions wanted" for each topic.



Major Features in OSVR-Core

Optical video-based (“positional”) tracking

The positional tracking feature uses IR LEDs that are embedded on the OSVR HDK along with the 100 Hz IR camera included with the OSVR HDK to provide real-time XYZ positional and orientation tracking of the HMD.

The LEDs flash in a known pattern, which the camera detects. By comparing the location of the detected LEDs with their known physical locations, a position and orientation (pose) determination is made.

The software current looks for two targets (LED patterns): one on the front of the HDK and one on the back. Additional targets can be added, and thus additional devices that have known IR LED patterns can also be tracked in the same space.

The image below shows a built-in debugging window that indicates the original image overlaid with beacon locations (in red, a tag of -1 means that the beacon has not been visible long enough to be identified) and reprojected 3D LED poses (in green, even for beacons not seen in the image). RANSAC-based pose estimation from OpenCV provides the tracking.

Centralized Display Interface

The OSVR-Core API now includes methods to retrieve the output of a computational model of the display. Previously, applications or game engine integrations were responsible for parsing display description JSON data and computing transformations themselves. This centralized system allows for improvements in the display model without requiring changes in applications, and also reduces the amount of code required in each application or game engine integration.

The conceptual model is “viewer-eye-surface” (also known as “viewer-screen”), rather than a “conventional camera”, as this suits virtual reality better.[1] However, it has been implemented to be usable in engines (such as Unity) that are camera-based, as the distinction is primarily conceptual.

As a demonstration of this API, a fairly minimal OpenGL sample (using SDL2 to open the window) is now included with OSVR-Core.

Render Manager

The Sensics/OSVR Render Manager provides optimal low-latency rendering on any OSVR-supported device. Render Manager currently provides an enhanced experience with NVIDIA’s Gameworks VR technology on Windows. Support for additional vendors (e.g. AMD, Intel) is being worked on.

Unlike most of the OSVR platform, the Render Manager is not open-sourced at this point. The main reason is that the NVIDIA API was provided to Sensics under NDA and thus we cannot expose it at this time.

Key features enabled by the Render Manager:

DirectMode: Enable an application to treat VR Headsets as head mounted displays that are accessible only to VR applications, bypassing rendering delays typical for Windows displays. DirectMode supports both Direct3D and OpenGL applications.

Front-Buffer Rendering: Renders directly to the front buffer to reduce latency.

Asynchronous Time Warp: Reduces latency by making just-in-time adjustments to the rendered image based on the latest head orientation after scene rendering but before sending the pixels to the display. This is implemented in the OpenGL rendering pathway (including DirectMode) and hooks are in place to implement it in Direct3D. It includes texture overfill on all borders for both eyes and supports all translations and rotations, given an approximate depth to apply to objects in the image.

Coming very soon:

Distortion Correction

High-Priority Rendering

Time Tracking .

Unity Low-level Native Plugin Interface .

Render Manager is currently available only for OSVR running on Windows.

Several example programs and configuration files for OpenGL and Direct3D11 are provided in open-source form as wel as a program with adjustable rendering latency that can be used to test the effectiveness of asynchronous time warp and predictive tracking as application performance changes.

Predictive Tracking

Predictive tracking reduces the perceived latency between motion and rendering by estimating head position at a future point in time. At present, the OSVR predictive tracking uses the angular velocity of the head to estimate orientation 16 mSec (1 frame at 60 FPS) into the future.

Angular velocity is available as part of the orientation report from the OSVR HDK. For other HMDs that do not provide angular velocity, it can be estimated using finite differencing of successive angular position reports.

Profiling tools

Utilizing ETW – Event Tracing for Windows – the OSVR performance profiler allows optimizing application performance by identifying performance bottlenecks throughout the entire software stack.

Event Tracing for Windows (ETW) is an efficient kernel-level tracing facility that lets you log kernel or application-defined events to a log file and then interactively inspect and visualize them with a graphical tool. As the name suggests, ETW is available only for the Windows platform. However, OSVR-Core’s tracing instrumentation and custom events use an internal, cross-platform OSVR tracing API for portability.

Currently the default libraries have tracing turned off to minimize any possible performance impacts. However, the “tracing” directory contains tracing-enabled binaries along with instructions on how to swap them in to use the tracing features. See this slide deck for a brief introduction on this powerful tool: http://osvr.github.io/presentations/20150901-Intro-ETW-OSVR/

JointClientKit

The default OSVR configuration has the client and server run as two separate processes. Amongst other advantages, this keeps the device servicing code out of the “hot path” of the render loop and allows multiple clients to communicate with the same server.

In some cases, it may be useful to run the server and client in a single process, with their main loops sharing a single thread. Examples where this might be useful include automated tests, special-purpose apps, or apps on platforms that do not support interprocess communication or multiple threads (in which case no async plugins can be used either). The new JointClientKit library was added to allow those special use cases: it provides methods to manually configure a server that will run synchronously with the client context.

Note that the recommended usage of OSVR has not changed: you should still use ClientKit and a separate server process in nearly all cases. Other ways of simplifying the user experience, including hidden/launch-on-demand server processes, are under investigation.

New Android Capabilities

A new device plugin has been written to support Android orientation sensors. This plugin exposes an orientation interface to the OSVR server running on an Android device. This is available here:

https://github.com/OSVR/OSVR-Android-Plugins

A new Android OpenGL ES 2.0 sample demonstrates basic OSVR usage on Android in C++. You can find this sample here:

https://github.com/OSVR/OSVR-Android-Samples

An early version of an Android app has been written that launches the OSVR server on a device to run in the background. This eliminates the need to root the phone, which existed in previous OSVR/Android version. You can find this code here:

https://github.com/OSVR/OSVR-AndroidServerLauncher

The Unity Palace demo (https://github.com/OSVR/OSVR-Unity-Palace-Demo/releases/download/v0.1.1-android/OSVR-Palace-Android-0.1.1.zip) for Android can now work with the internal orientation sensors, as well as with external sensors

New Plugins and Interfaces

Gesture interface

The gesture interface brings new functionality that allows OSVR to support devices that detect body gestures including movements of hand, head and other parts of the body. This provides ways to integrate devices such as Leap Motion®[2], Nod Labs Ring, Microsoft® Kinect® [3], Thalmic Labs™ MYO™[4] armband and many others. Developers can combine gesture interface with others to provide meaningful information such as orientation, position, acceleration and/or velocity about user's body part(s) pose.

Locomotion interface

The locomotion interface adds an API to support a class of devices also known as Omni-Directional Treadmills (ODT) allow walking and running on a motion platform and then converts this movement into navigation input in a virtual environment. Some examples of devices that would be able to use locomotion interface are: Virtuix Omni™, Cyberith™ Virtualizer, Infinadeck™, and others. These devices are very useful for First Person Shooters (FPS) games and by combining locomotion interface with tracker additional features such as body orientation, jump/crouch sensing could be added.

EyeTracker interface

The EyeTracker interface provides an API to report detailed information about the movement of one or both eyes.

This includes support for reporting:

3D gaze direction - a unit directional vector in 3D

2D gaze direction - location within 2D region

Detection of blink events

SMI Eye Tracker plugin

In collaboration with SensoMotoric Instruments GmbH (SMI) we are releasing a new plugin for SMI trackers. For instance, the plugin supports the SMI Upgrade Package for Oculus Rift™ DK2. It uses the SMI SDK to provide real-time streaming of eye and gaze data and report it via EyeTracker interface.

The SMI plugin also provides an OSVR Imaging interface to stream the eye tracker images.

Simulation plugins

Along with the newly added interfaces (eyetracker, gesture, locomotion), we provide simulation plugins that serve as an example on how to use a certain interface. Their purpose is emulate a certain type of device (joystick, eyetracker, head tracker, etc.), connected to OSVR server, and feed simulation data to the client. These plugins were added as a tool for developing applications so that developers can easily run tests without the need to attach multiple devices to the computer. We would be expanding the available simulation plugins to have one for every type of interface. Simulation plugins are available in OSVR-Core and can be modified to a specific purpose.

Release Notes

OSVR 0.6 has many other changes, and improvements. See the complete list here

In Closing…

As always, the OSVR team with the support of the community is continuously adding smaller features and addressing known issues. You can see all of these on Github such as in this link for OSVR Core