Before we continue, let’s understand what all these different file formats mean.

What’s the difference between USD, USDC, USDA and USDZ?

Files with .usd , .usdc and .usda extensions all represent a USD scene file, but the way they store this data is different.

Recall that a USD file can be thought of as representing a scene, and a USD scene is a hierarchy of nodes. Most of the time you will have a 1-to-1 correspondence between a USD file and a 3D model, instead of a scene; i.e. you will have one single mesh in the file.

.usda

.usda files are USD scenes stored on file in ASCII format. Hence, these files are text files containing the scene description in a manner that is human-readable. For example, the following is the .usda file for the representation of a USD scene containing a simple cube:

#usda 1.0 def Mesh "cube"

{

float3[] extent = [(-0.5, -0.5, -0.5), (0.5, 0.5, 0.5)]

int[] faceVertexCounts = [4, 4, 4, 4, 4, 4]

int[] faceVertexIndices = [0, 1, 3, 2, 2, 3, 5, 4, 4, 5, 7, 6, 6, 7, 1, 0, 1, 7, 5, 3, 6, 0, 2, 4]

normal3f[] normals = [(-0.15186, 0.988312, 0.013374), (-0.206543, 0.977774, -0.036018), (0.026661, 0.999633, 0.004783), (0, -1, 0), (1, 0, 0), (-1, 0, 0)] (

interpolation = "uniform"

)

point3f[] points = [(-0.5, -0.5, 0.5), (0.5, -0.5, 0.5), (-0.5, 0.5, 0.5), (0.5, 0.5, 0.5), (-0.5, 0.5, -0.5), (0.5, 0.5, -0.5), (-0.5, -0.5, -0.5), (0.5, -0.5, -0.5)]

uniform token subdivisionScheme = "none"

}

As you can see the format is quite readable and makes it easy to understand how the scene is organised.

However, this format is not efficient as it takes a long time for large files to be parsed into a text file. For this reason, Pixar also specifies the .usdc file format, which allows software to read USD scenes much more quickly.

.usdc

.usdc files are the binary representation of USD scenes. Hence, they are stored in binary form, and opening them with a text editor is not of much help. Here is the cube scene example again, this time after opening its .usdc equivalent with a text editor:



��� ��� �����������������¿���¿���¿���?���?���?����������������������������������������Ô���EEEUU�ÿ�úÿþþú����������������������?������?��������������¿������¿������?����������¿������������������¿���¿���?���?���¿���?���¿���?���?���?���?���?���¿���?���¿���?���?���¿���¿���¿���¿���?���¿���¿�������Ú�������Ö��������ó�primChildren�cube�specifier�typeName�Mesh�properties�subdivisionScheme�points�extent�normals�faceVertexCounts�faceVertexIndices�float3[]�default�int[]B�ò 3f[]�interpolation�uniformq� �ðtoken�variability�none������������������������ð

���UEQþö�öóöùk�������� X��)�a���*@�1

�1 @�1�1 @¨�2È�! �1 @ø�1�3 @� H �0�1� @�°�,@����� @���������������ð

���EUP�ÿüúø÷

ô

òï�������������� ��������p�����

��������°���QA�õþÿ ������������@ÿþ������� ��������p����� �������� ���� �������������ÿû�������TOKENS����������°������î�������STRINGS����������������������FIELDS����������¦�������������FIELDSETS�������B������/�������PATHS�����������q������I�������SPECS�����������º������@������� PXR-USDC�������ú��������������������������������������������������������������������������������������������� ������ ��� �����������������¿���¿���¿���?���?���?����������������������������������������Ô���EEEUU�ÿ�úÿþþú����������������������?������?��������������¿������¿������?����������¿������������������¿���¿���?���?���¿���?���¿���?���?���?���?���?���¿���?���¿���?���?���¿���¿���¿���¿���?���¿���¿�������Ú�������Ö��������ó�primChildren�cube�specifier�typeName�Mesh�properties�subdivisionScheme�points�extent�normals�faceVertexCounts�faceVertexIndices�float3[]�default�int[]B�ò 3f[]�interpolation�uniformq� �ðtoken�variability�none������������������������ð���UEQþö�öóöùk�������� X��)�a���*@�1 @d �1)��1 @�1�1 @¨�2È�! �1 @ø�1�3 @� H �0�1� @�°�,@����� @���������������ð���EUP�ÿüúø÷òï�������������� ��������p�������������°���QA�õþÿ ������������@ÿþ������� ��������p����� �������� ���� �������������ÿû�������TOKENS����������°������î�������STRINGS����������������������FIELDS����������¦�������������FIELDSETS�������B������/�������PATHS�����������q������I�������SPECS�����������º������@�������

However, these files can be read by softwares much more quickly than their .usda equivalents. In fact, USDZ packages for QuickLook must use .usdc for the scenes to be loaded quickly on mobile devices.

.usd

.usd files can be both ASCII-based or binary files. Their extension doesn’t give away what format they are from the outside, but systems reading them can determine their format from the first bytes of the file. Hence, .usd files can be both text-based or binary based, i.e. you can rename either a .usdc or .usda file to .usd and they will work like how they were working before you renamed them.

.usdz

This format is used by QuickLook to render scenes and can be imported and used in iOS projects developed in Xcode.

In its current form, it’s more or less a zipped folder containing at most (and at least) one .usdc file and any textures which that scene uses.

You can learn how to create USDZ files from OBJs here:

Exploring the Stratocaster USDZ asset

Now that you have extracted the Strato, you can convert its .usdc file to .usda as such:

cd stratocaster/

usdcat Stratocaster.usdc -o Stratocaster.usda

Open the Stratocaster.usda file with your favourite text editor, and boom:

The beginning of the Stratocaster.usda file

We can now read the contents of the file to understand how USDZ assets are organised for consumption by QuickLook.

Open the newly created Stratocaster.usda by double clicking on it so that the changes we made to it are shown in real-time.

Materials

Just at the top you can see the definition of the various materials for the asset. Some of these use constant properties, whereas the others, such as this one, use a texture map:

The material for the body of the Stratocaster, which uses a texture

For usage with QuickLook, the names of the attributes (a.k.a. the material properties) in the various pbrMat1 need to be exactly the same as shown in the example above. Changing the name of diffuseColor to diffuse in the material “StratocasterGuitar_SynthString_Material” (not the one shown above) results in the loss of albedo colour for the components using that material:

Notice the strings on the left: they are missing albedo information

For every material attribute that uses a texture instead of a constant value, QuickLook expects a material Prim like this one:



{

uniform token info:id = "UsdUVTexture"

float4 inputs:default = (0, 0, 0, 1)

asset inputs:file =

float2 inputs:st.connect = </Materials/StratocasterGuitar_NeckWood_Material/Primvar.outputs:result>

token inputs:wrapS = "repeat"

token inputs:wrapT = "repeat"

float3 outputs:rgb

} def Shader "color_map"uniform token info:id = "UsdUVTexture"float4 inputs:default = (0, 0, 0, 1)asset inputs:file = @StratocasterGuitar_N eckWood_Albedo.jpg@float2 inputs:st.connect = token inputs:wrapS = "repeat"token inputs:wrapT = "repeat"float3 outputs:rgb

and you also need to provide a texture sampler:

def Shader "Primvar"

{

uniform token info:id = "UsdPrimvarReader_float2"

float2 inputs:default = (0, 0)

token inputs:varname = "Texture_uv"

float2 outputs:result

}

This is what the material definition looks like when taken as a whole:

Another material making use of a texture map for one of its properties.

Scene Graph and Meshes

Expanded view of the scene graph.

Looking at the expanded Scene Graph for the Stratocaster model, one thing is immediately noticeable: seems that elements are repeating.

What’s the difference between the “splitContainer” section and the elements ending with “Material”?

tl;dr: They are essentially the same thing. And the nodes without anything attached, the ones ending with “Material”, are simply a by-product of how USD assets are parsed by the viewer and SceneKit.

If we go back to our .usda file, we can look for the text “StratocasterGuitar_ButtonTone_Material”, and here it is:

The ButtonTone submesh

Submeshes are represented by GeomSubset Prims in USD. They are used to specify subsets of the whole model to which to apply a material exclusively.

If we try to look for any of the “splitContainer” elements, we wouldn’t find them in the file. It is because they are auto-generated from the indices of a given submesh, and they use the material they are assigned via the rel material:binding relationship.

The indices listed above for the ButtonTone element are indices into faces, instead of vertices. They refer to the global face indices for the whole guitar mesh, which can be found further up in the file:

Each element in the array describes how many vertices that face uses.

The “StratocasterGuitar” Prim also defines the elements composing a vertex: positions, normals, texture coordinates, and etc, which are then indexed via vertex indices.

Materials in USD

The USD specification doesn’t explicitly forbid the usage of non-PBR materials, but iOS’ QuickLook doesn’t work with non-PBR materials. So, make sure that your USDZ file uses PBR materials or QuickLook will not render them properly.

Why is PBR important?

PBR stands for “Physically Based Rendering”. PBR enables a variety of enhancements on how realistically objects can be rendered. One of these is the ability to use image-based lighting, which provides more accurate ambient diffuse and specular reflections.

The differences between “traditional” shading and PBR shading can be stark:

You can learn more about PBR from here and here.

In Augmented Reality, PBR is especially relevant as it allows the 3D models being rendered to be shaded in a way which is more aware of its surroundings. For example, if you were to render a USDZ model of a shiny metallic cube, with PBR materials and QuickLook the cube would reflect the environment around it as if it were a real one:

Realistic reflections on a metallic cube with ARKit

Current Issues and Limitations with USDZ

Toolset

One of the biggest pains by far is the lack of good tools for artists to export their creations from their asset creation software to USDZ directly and without having to use the command line tools. Command line tools are not for everyone, and most of all the provided ones work with only a handful and outdated intermediate asset formats, making it hard to really exploit the functionalities of the USD format.

Interactions

In their current form, neither the USD or USDZ specifications talk about the ability for the user to interact with the 3D scene described by a USDZ file. Hence, all that’s possible to do at the moment when opening a USDZ file in QuickLook is to look at it, and nothing else. For example, tapping on it won’t trigger any actions and such actions has no way to be coded into the format anyway.

You can bake animations into the asset, but those are pre-baked and not interactive.

PBR only

You can’t use non-PBR materials when combining USDZ with QuickLook. Note that nothing stops you from declaring Attributes (in USD-speak) on your Prims which have names resembling the material properties you’d find on, say, a Lambert material, but they won’t work with Pixar’s renderer or Apple’s.

This is a limitation especially when you want to display widgets/abstract 3D objects rather than 3D models of realistic objects.

Apple products only

So far USDZ hasn’t been adopted by Android or other platforms, and hence it can only be consumed by users with iOS and MacOS devices. On other platforms, gLTF and gLB seem to be the main candidate to become the USDZ equivalent.

Moreover, QuickLook is only available on iOS. Although third-party solutions exist, there isn’t an equivalent native solution on Android. But you can be sure that if and when it arrives, it will use the gLTF format and will display the content slightly different from QuickLook.

My predictions

AR is inherently averse to the concept of apps: you don’t want to switch to a different app every time you want to access a different AR layer or service, let alone download and install an app which you’re going to use only when at a specific location.

For this reason, the fact that USDZ allows users to interact with AR content without having to install an app is a step forward in that direction, albeit a small one. To really move in the direction of an AR cloud, we need to be able to, at least, add interactions to experiences such as the ones offered by USDZ.

This calls for a solution closer to a web browser than an app store; Mozilla and others are working towards that solution with WebXR.

Given what the USD format is intended as a format to represent 3D data, but not interactions, I don’t think that Apple or Pixar will modify the specifications to introduce interactions and scripting.

Instead, Apple will either have to bite the bullet and allow users to work with something similar to the WebXR/AR or extend QuickLook/Safari to integrate scripting of the USDZ scenes available.

Apple will have to gradually introduce better tools to author USD assets if it doesn’t want to officially support gLTF. So, we might see some announcements with respect to that at WWDC 2019.

References

https://graphics.pixar.com/usd/docs/

https://graphics.pixar.com/usd/docs/USD-Glossary.html

https://graphics.pixar.com/usd/docs/Usdz-File-Format-Specification.html

Conclusions

USDZ, when combined with ARKit and QuickLook allows you to:

let users try out 3D models of realistic objects without having to install apps (which is great for e-commerce applications);

send 3D models to friends via iMessage;

package USD scenes into a portable format; and

render objects in 3D realistically thanks to PBR;

By far the most promising usage for USDZ files is the first one on the list: users can try out products in real-life scale and with a lighting environment that makes them look the same as the final product as much as possible: