New Unity Asset Lets You See Your Actual Hands — Not Just a Rigged Replica

Using our existing hand assets, you can already reach into a demo and see robot hands, minimal hands, even realistic hands for different genders and skin colors. But these aren’t your hands – it’s like choosing from a catalog of hand models. That’s why we’ve just released a new Unity asset feature that brings your real hands into any VR experience. With Image Hands, you now have the absolute realism of live video, with the full interactivity and 3D behavior of our classic rigged hands.

The Image Hand is available now as part of our Unity Core Assets. You can also try it yourself in our updated UI Widgets demo and the latest version of ElementL: Ghost Story, or learn more from the official documentation.

DOWNLOAD THE DEMO

Why Your Hands?

We believe that augmented reality and virtual reality are just waypoints along a spectrum of possible experiences. It starts with pure reality, which can be augmented with virtual objects that respond to real-world actions. When every real object has been hidden, you’ve entered virtual reality.

Our Unity assets already make building these types of experiences possible with virtual rigged hands. However, we’ve found that users sometimes reported feeling disconnected from the existing virtual hand assets. Plus, even a small tracking jitter can have an enormously negative impact on the experience.

Our Unity assets already give you access to a wide variety of virtual hands.

On the augmented side of the reality spectrum, developers already have access to the controller’s raw infrared video with our Image API. But since these hands are just flat image textures, you were limited in terms of what you could create.

Our Image Hand next-gen software demo brings these two worlds together by projecting the raw images of your hands into a 3D mesh that can interact with other objects in real-world space. The part of the mesh that you can see is covered by the passthrough from the twin Leap Motion cameras. Each camera provides a separate view of your hands, so that what you see has actual depth.

This hybrid effect has a powerful impact in VR because your real hands can now actually pass through (or disappear behind) virtual objects. The hands interact properly with other 3D objects because they are 3D – complete with the interactive and visual capabilities that you expect. This approach also reduces jitter effects, since there’s no longer an artificially generated rigged hand to worry about. While the hand image in VR might shift or become transparent, it won’t suddenly shift in the wrong direction.

The Image Hand can pass through virtual objects with full 3D fidelity.

Glowing with Confidence

The Image Hands are also designed to provide users with dynamic feedback on the Leap Motion Controller’s tracking confidence. When your hand assumes a high-confidence pose, it will glow blue. The glow disappears as confidence values drop. Developers can also use this feature as a form of quick and easy testing while prototyping.

Similarly, we’ve also added a glowing depth cue. Whenever your hand touches or penetrates a virtual object, a highlight is applied to the intersection point. This makes it easy for users to know where everything lies in 3D space.

Getting Started

We’re excited to release this tool to the community and see how you use it in your own projects. Whether you’re augmenting reality, creating a virtual world, or somewhere in between, we’d love to hear about your project. Shoot us your feedback in the comments or on our Unity assets forum thread.

Update: For maximum hand presence, be sure to follow the steps in the video below. You can learn more on this forum thread.