Asset Pipeline: Animating 3D Characters for AR/VR

Step by Step guide to animating 3D assets for use in AR/VR

Here’s a handy guide on how to add animations to 3D characters and import them successfully into your AR/VR projects. I’ll show you some best practices and go over my 3D animation process.

If you want to learn the basics of 3D modeling or import just a static 3D object, I suggest checking out my previous guide here.

Software

This is a list of what I currently use to produce animated 3D characters. If you prefer other software packages, by all means use what you’re comfortable with. Just follow the guide to make sure you’re exporting the 3D model correctly.

Autodesk Maya (Free Trial) — Industry standard 3D modeling software for many years but no longer the king of the hill. There are many other software packages I use depending on the type of model I make, but this has a great exporting tool (Game Exporter) which has made it easy for me to export animated FBX characters that work well with rendering platforms.

Fuse (Free for Adobe CC members) — Quickly Design custom 3D characters using sets of pre-made components.

Pixelogic Zbrush ($795) — Industry standard 3D sculpting tool I use to create a high-poly base mesh for characters or complex objects.

FBX Review (Free) — A lightweight standalone tool that lets you view your exported FBX files. If it shows up properly here, it will show up properly in your rendering platform.

Mixamo (Free) — Quickly rig and animate your character for free using just your browser. Manually rigging a character is tough, time-consuming, and requires many hours of practice to get decent. Luckily, Adobe provides Mixamo, and with just a few clicks, does all the hard work for you. Not only does it rig, but you can apply one of many pre-made animations to your character. Note: Mixamo only works with standing bipedal humanoid models. Quadrupeds and snakes will not rig.

Viro Media (Free) — developer platform for building AR/VR applications. Great tool for visualizing assets in AR or VR and for iterative AR/VR design. (Detailed instructions below)

Best Practices

It doesn’t matter what you use to create or get your 3D character, it just needs to be optimized for real-time use and exported to an OBJ or FBX format, the most widely supported formats. I prefer exporting to FBX, because you can embed multiple animations and yields the most consistent and predictable results.

3D Resources

If you’re not creating a 3D animated character from scratch, there are several great sites that have quality realtime characters to download:

www.sketchfab.com | www.turbosquid.com | www.poly.google.com|www.free3d.com | www.cgtrader.com | www.gumroad.com

I also frequently post free 3D assets for use in AR/VR apps here -> Viro 3D Assets

Real-Time Specs

Polycount — I like to make sure my character model is not overly tessellated and running over 50k polys. On mobile, an animated character that is over 200k polys will most likely effect framerate and limit how much you can put in your scene.

— I like to make sure my character model is not overly tessellated and running over 50k polys. On mobile, an animated character that is over 200k polys will most likely effect framerate and limit how much you can put in your scene. Texture Maps — Make sure it’s 1 texture per object. I’ve ran into issues when I assigned multiple textures to 1 object. For instance, if your character has separate texture maps for the body and head, make sure they are separate objects and not combined into 1 when exporting. Make sure texture maps are in .PNG or .JPG format.

— Make sure it’s 1 texture per object. I’ve ran into issues when I assigned multiple textures to 1 object. For instance, if your character has separate texture maps for the body and head, make sure they are separate objects and not combined into 1 when exporting. Make sure texture maps are in .PNG or .JPG format. Skinning Skeleton — If manually binding skin to a skeleton, make sure max influences is set no higher than 4.

— If manually binding skin to a skeleton, make sure max influences is set no higher than 4. T-Pose — Make sure your character is in the standard T-Pose. When importing your character into Mixamo, I found this resulted in the most predictable results.

Create or Download 3D Character

Retopologize using ZRemesher

For this guide, I created a Cartoon Turkey as our sample asset. You can grab it here. (Want the Maya file? Grab it here.)

When creating a character from scratch, I start with sculpting in Zbrush, then use Geometry>ZRemesher to quickly retopologize and create a low-poly real-time version. This isn’t the most efficient way to create a low-poly character, but when you’re pressed for time, it provides fast results and generates nice, evenly-spaced quad-based topology. I then follow these steps to UV Unwrap my low-poly model, bake, and paint textures.

Turbosquid Filtered Results

When purchasing/downloading a pre-made model from sites like Turbosquid, I usually filter my results to “Real-Time” and “10K to 50K” polys. After downloading the model, I import the model into Maya to double check and make changes as needed to follow the real-time specs defined here. I then export the static T-pose character as a .FBX.

Note: If your model is already rigged, for this tutorial, I suggest detaching the skin off the skeleton and making sure the character is in a T-pose.

Rig and Animate Character

Mixamo Auto-Rigger

Now that my turkey character model is ready for rigging, I use Mixamo.

Go to www.mixamo.com. Login with your Adobe CC account. Upload .FBX character model and once loaded, it will display the Auto-Rigger tool. Drag and place the markers on the appropriate spots on your character then hit “Next.” It will take a couple min, but if your character is successfully rigged, you’ll be able to see it animating within the same window. Hit “Next” if it looks good. You can now apply any pre-made animation loop to your newly rigged character. I wanted the turkey to dance, so I filtered the results to show only dancing animations. Adjust animation using sliders in the right menu. For instance, the hands were clipping into the turkey’s body so I raised character arm-space. When you find animation you want, hit the “Download” button on the right menu bar. I want to add 1 more idle animation to my character, so I download that as well. You browser will download the animated character as a .FBX file.

Apply an animation to your rigged character

Set up Multiple Animations in 1 FBX

All joints selected, all keyframes selected, and copied

I want to have multiple animations for my character model: 1 dance, and 1 idle. I need to set them up both in 1 .FBX file. How do I do this? This is the easiest way that worked for me:

First, I import the .FBX with the dancing animation into Maya. Then I select all the joints. Edit>Select All by Type>Joints. You should see red keyframes appear on the bottom timeline. Select all the keyframes, right click then click “Copy.” Note the keyframe length of the animation. Then, I open the .FBX with the idle animation in Maya. This will completely replace the scene. Next, I need to define where the idle animation starts and ends when I export. Open Game Exporter: File>Game Exporter. Click on “Animation Clips” tab. Click on the green plus icon. There should be open fields for Clip Name, Start, End. I label the field under “Clip Name” to 01. Then I fill in 0 under “start” and 208 under “End,” because that’s when the idle animation ends. Next, I click on the green plus icon underneath the fields. This adds a new row to fill out for my dancing animation. I label the new “Clip Name” to 02. Then, I define the starting keyframe of the dancing animation. I like to add a little padding between both, so I start it at 215. For “End,” since I noted the length of the dancing animation in step 2, I calculated it to end at 502. Next, click on the film strip icon next to 502. This will focus only on the keyframe segment you defined for animation 02. Next, I select all the turkey’s joints again: Edit>Select All by Type>Joints. With all the joints selected, I click on frame 215 in the bottom timeline. Right click>Paste>Paste. This will apply the dancing keyframes onto the same bones to this clip. If you pasted the keyframes correctly, you should have both animation clips in 1 file now. Click on the filmstrip icon next to each clip to focus on each animation and hit play to view them.

Export Character

Game Exporter

Now that you have both animations in 1 file, it’s time to export to .FBX.

Under “Clip Settings,” Make sure “Bake Animation” is checked. Under “Clip Settings,” click on the drop down and choose “Save Multiple Clips to Single File.” Choose directory and save file to under “Path.” I name my file turkey man_anim Click “Export” button. A popup should confirm successful export.

Test Character in FBX Review

I like to verify my exported models in FBX Review. It’s a quick way to check the model is displaying and runs animations correctly.

Open the newly exported .FBX model. It should default to clip 01. Click play button on the bottom controls to play animation. To switch to clip 02, click on top right dropdown to select 02. Click play button to watch second clip. If both are playing properly, congrats, your character is ready to be used for your AR/VR app!

Visualizing 3D Assets AR/VR

This next section shows you how to view your assets in AR using the Viro platform (To view your assets in VR using Viro, follow the steps below starting with HelloWorldScene.js rather than HelloWorldSceneAR.js). While Viro is a platform for developers, it can also be used by designers and creatives to visualize 3D assets and iterate on ideas. Set up is simple and the steps listed below are the same one’s I use to view 3D characters in AR/VR as a designer.

Setting up Viro

Use our Quick Start guides to get setup with Viro in minutes. The Viro platform is free with no limits on distribution. Sign up to get your API key and start developing AR apps today:

You will also need either an iOS ARKit enabled device or an Android ARCore enabled device.

Java developer? Get started here.

Once you complete the setup step and see Hello World on your device, you should move onto the next section to display your 3D assets in AR.

Displaying 3D Assets in Viro

We will be using our Dancing Turkey model for this next section to simplify the steps.

First you need to convert our FBX files in VRX format by using the ViroFBX script. The ViroFBX script can found in your project’s bin directory.

Copy your turkeyman_anim.fbx and the associated texture files into a folder and name the folder -> turkeyman_anim. Copy the folder into your bin folder which is located within the ViroSample folder.

Open Terminal on your Mac and go to your bin directory. I setup Viro in a folder called “WorkSpace” so I would enter the command below (replace WorkSpace with your root folder in the command below):

cd WorkSpace/ViroSample/bin

Once you are at your bin directory, enter the command below.

./ViroFBX turkeyman_anim/turkeyman_anim.fbx turkeyman_anim/turkeyman_anim.vrx

Once the script is completed, you should see a file called “turkeyman_anim.vrx” in your turkeyman_anim folder in your bin directory.

Once the VRX file is created, it can be loaded into an application with <Viro3DObject> . Copy your turkeyman_anim folder and paste it into the res folder located at /ViroSample/js.

To see your animated character in AR, open your HelloWorldSceneAR.js file and replace the text in your HelloWorldSceneAR.js with the code below.

Save your HelloWorldSceneAR.js file. Shake your device to bring up the menu and tap “Reload”. You should see the Dancing Turkey in front of you below “Hello World”. We added a shadow to the character and enabled drag so you can move the character around with your finger. You can also tap on the turkey to pause/unpause the animation.

To see your own assets in AR:

Convert your FBX file into VRX format. Place your VRX assets in the correct folder/path. Update your code for the new asset. In the example above, you would change “turkeyman_anim.vrx” to your vrx file name and change the texture names (lines 49–52). Make sure the name of your animation has been updated in line 57. The dancing animation for turkeyman is called “02” Save your file and reload to see your 3D asset in AR.

Questions? Problems? Need extra help importing your 3D object? Send me an email. I’ll be glad to help.