I’ve always preferred story-driven games to the competitive ones. When diving into a game like this, one experiences a variety of emotions and may also grow as a person at the end. Despite all that I came to the realization that we might control these characters — our avatars in a way — , but in effect, they have nothing to do with us!

When playing a game, you see the body and hands of your playable avatar instead of your own. You can customize your character in some MMORPGs, but it still requires heavy manual tweaking to make them look like you.

This is even more relevant in virtual reality, where the control is, literally, in the player’s hands. Hand-object interaction is in the center of a great virtual reality experience.

Seeing someone else’s hands move instead of your own feels uncanny and causes discomfort. Although this sounds like a small detail, it can really break the immersion, especially in VR. To work around this, most games apply clever tricks, like covering the player’s hands or showing gloves instead.

However, hands differ in shapes and sizes, so these workarounds only offer partial solutions to the problem.

Hands in The Gallery — Episode 1: Call of the Starseed

With my project, I aim to change the character’s body to resemble the player’s physical image in order to make gaming a truly personal experience, thus making it more immersive.

I want to transform the character’s body to resemble the player’s own physical characteristics, making games a truly personal experience.

Recent advancements in Artificial Intelligence made it possible to take personalization to the next level, whether generating voice on the fly (TTS) or modeling the player’s face from a single image.

Online Demo

Take a look at my interactive demo here. You can change the shape of the hand by adjusting the sliders on the right.

Try finding the right parameters for your hand, let’s see if you can.😉

Leave a comment with your results.

Deformable Hand Demo

The Model

As a starting point, I’m using the Mano model. It is a parametric hand model published by the Perceiving Systems Department in Tübingen.

I’ll get into more details in the following article. If you are interested, make sure you’ve clicked the Follow button next to my name.

Researchers from PSD published all the data accompanying their high-quality paper which is still very rare these days.

There is a catch, however. They still use Python 2 and therefore the data is in a Pickle format incompatible for Python 3, that I’m currently using. Furthermore, the structure of the files was “documented” only by an example code. I had to figure out the purpose of each matrix by their dimensions and arbitrary names.

All in all, I came to the conclusion that I needed to export the values as JSON files to maintain cross-compatibility between various programming languages.

Skeleton with Blend Weights

Based on the published files I’ve rebuilt the whole model in Maya to be able to export to different game engines.

I managed to set up the blend shapes by hand but that was not an option for the rig. I would have had to manually set blend weights for every one of the 778 vertices for every 16 joints.