Dislacement

Vertex displacement is the best. It lets the Shader drive the position of each vertex on the mesh, which can give you this kind of result:

I’m going to start right away with a demonstration. We’ll be making a wobbly sphere, like the example above.

Make a sphere in your scene, and add a “Graph PBR” material to it (which you can make in the Resources panel).

The Graph PBR is simply a preset of a Shader Graph material that already has the right nodes for the main textures, so we can skip the boring stuff and get straight to displacement.

You should start off with this preset of a sphere:

Open up the Shader Graph panel. Move all the texture stuff out of view – we need some space to work. Do keep it connected, though.

As you can see in the Shader node, there also is a World Position (Vertex) input. That’s what we’re going to be working with.

Since these are absolute world position values, it means the values are not relative to the mesh. For instance, giving it a value of 0 will make the object invisible (since all vertices will then be at the same position: [0, 0, 0]).

To make the mesh wobble, we will need the current vertex position first. The wobble needs to be added on top of that.

You can get the current vertex world positions with the node called ‘Surface Position’. Connect it to the World Position input in the Shader node, and nothing will change. Which makes sense – you’re giving it the same information it already had.

Now we can add any values to this Surface Position we like!

For instance, if you make an Add node, connect it to this line, and connect an ‘Elapsed Time’ node to the second input of the Add node:

The sphere will now slowly move up, to the right and towards you. In a diagonal direction. Change the Multiplier parameter in the Elapsed Time node to make it go faster or slower.

How does this work? And why is it moving diagonally?

The Surface Position data has 3 dimensions: x, y, and z. If you add one-dimensional data from a node like the Elapsed Time node (which outputs a number in seconds) to all of those values, they will all go up.

A positive number on the X-axis is going to the right, one on the Y-axis is moving up, and one on the Z-axis is pointing towards you.

If you get all three at once, it means it will diagonally point in the combined direction of up, right, towards you.

So to make it move away, to the left, and downwards, diagonally, you’d simply have to multiply the Elapsed Time node by -1.

The reason I am explaining this behaviour, is that it will be coming back later. It’s a side effect of working with three dimensions.

If you want it to move on the X axis only (left to right), you could add a ‘Combine’ node between the Elapsed Time and the Add.

Only connect it to the x input of the Combine, so the y and z will stay at 0. The ‘w’ can be ignored for now.

This way, only the X-axis will receive that value from Elapsed Time. The other two will remain at their original position!

Now we’re going to generate that wobbly randomness, using a Voronoi noise node.

The Voronoi Noise node generates a random pattern, based on the seed you give it. In the case of 3D models, you can give it the UV map of the model!

Temporarily connect the Voronoi Noise node to the Shader color input – that way you can see what’s happening!

Also, connect the Seed input to a ‘Surface UV Coord’ node, which gives the object’s UVs.

That’s looking pretty good. You can even connect that Elapsed Time node to the Offset input of the Voronoi Noise, so it will be animated!

Of course, we will be using this in the Vertex Displacement later. But for now, let’s keep it visible by having it connected to the Shader color.

The next step in making a wobbly sphere is making this noise work in both directions.

We want the wobbles to go inwards and outwards the sphere, which means the noise values will have to go into negatives as well.

Right now, as is the default, the Voronoi Noise is only giving values between 0 and 1 (black and white). We want it to be -1 to 1, so it moves in both directions.

To remap these values, simply add a Remap node and set it to Range In: 0 to 1, and Range Out: -1 to 1.

Now, the noise is ranging from -1 to 1, which we can’t really see in the Preview, since 0 and everything below it is shown as black.

Let’s reconnect the PBR node that was already there to the Shader color input.

Now, let’s add that Surface Position node to the output of the Remap node (so the noise is added to the mesh), and connect that result to the World Position (Vertex) input of the Shader node!

The result in the preview is barely visible, but it is there. To make it more extreme, add a Multiply node with any higher value in it (I use 30) directly after the Remap node (before the Add node):

Now you can really see some animated displacement!

The last thing we need to do, is to get rid of that diagonal direction in the displacement.

This is, like with the example I showed earlier, caused by having the same value added in all three axes. In this case, that value is the Voronoi noise.

It is the same in all three directions, so let’s create three different versions for it!

Copy the Voronoi Noise two more times, and connect the Surface UV node and Time Elapsed nodes that were already there to those new ones as well.

Next, we’re going to need some variation. For the second Voronoi Noise, place an Add node between it and the Elapsed Time node. Enter some random, high value (like 1000).

Now do the same for the third noise, but make it even higher (2000).

All three noises will now be different values, but how do you join them together into one stream of data that we can plug into the Remap node?

That’s what a ‘Combine’ node is for. Plug all values in their respective input in the Combine, and you’ve got it bundled up for Remapping. And that’s it!

This, by the way, is a basic version of what’s going on in that Triplanar Noise subgraph you can find in all those preset materials from the Material Library!

You can actually preview this noise, which is a cool looking effect in itself. Temporarily connect the output of the Combine node to the Color input of the Shader node, and you’ll see this colorful, animated texture.

I turned down the value in the Multiply node a bit, too, just to make it look better.

Now that the displacement works, we want it to be more customizable! For instance, the Scale input (xy) in the Voronoi Noise nodes should be exposed in the Inspector, so you can play around with the scale (in both directions, x an y, separately).

To do that, add a Float Parameter and set it to Channels: xy, Title: scale,

and connect it to all three Scale inputs of the Voronoi nodes. You can now change the noise scale from the Inspector!

Keep in mind that this Lens Studio default Sphere object only has a certain amount of vertices in it – if you want a higher resolution for displacement, you’ll have to make your own mesh in another 3D software.

Or download my high res Sphere object down below.

A good exercise would be to try and make another parameter that gives you the option to set the overall noise displacement amount!

It’s not a very difficult challenge, but it’s a good check to see if you understand how this works. I have added this parameter to the download for this project file down below!

In this download, the Sphere mesh is also of a much higher resolution to make the distortion more visible. This mesh can be downloaded separately at the bottom of the page as well.

This distortion method is fine for most meshes, but if you are working with a sphere, you will see a hard line in your displacement (depending on the scale of the noises):

To learn about why this is happening – and how you can fix it – check out the section Triplanar noise & Voronoi on spheres.

In that section, I will explain what the Triplanar Noise subgraph does, and I will demonstrate how to make your own version of it that uses Voronoi noises.

It will be slightly more complex, so you’ll need to understand this section first before diving into that one.

This new Triplanar Voronoi subgraph will also be included as a material in the download for this wobbly sphere!

This is what the seamless, more organic Triplanar Voronoi noise looks like:

﻿

Please keep in mind that there are many ways to get a cool looking distortion effect. You don’t have to closely follow this tutorial – in fact, in the case of a spherical mesh, it is probably better to not have three different Voronoi Noises and instead multiply one noise value by the Surface Normal directions!

This way, the mesh will still be wobbly, but only in the direction of the normals. Which makes it look more like it’s boiling and less like it’s just randomly moving around.

You could also use a Simplex noise instead of a Voronoi one, which will give you a result closer to the ‘Jello’ material in Lens Studio’s Material Library!

I have made an example PBR Graph Editor material that you can download at the bottom of this page,

it’s called ‘Displacer’ and it is like a standard PBR material except it has a bunch of Vertex Displacement options in it!

Opening it in the Shader Graph is also a great way to learn if you’re new to it.

This is what Displacer’s material options look like:

All parameters in this material are easily accessible via script. Using a simple script to modify the Distortion along Normals could, for instance, be used to make this ‘balloonify’ setup (project file download at the bottom of the page):

Projections

And now for something completely different: projection distortion.

I will be demonstrating how to make the following surprisingly simple effect

(again, project files can be downloaded at the bottom of this page).

This effect is setup the following way:

First, there’s a plane with a high vertex count.

The Device Camera Texture (the video coming from the user’s phone) is then ‘beamed’ onto that surface, from the perspective of the virtual camera.

Then, the plane is distorted while still having this texture on it, and rendered on top of the camera input.

Let’s start with step 1! Go get that high resolution geometry. The default Lens Studio plane is not good enough, since it only has 4 vertices. We’ll need something closer to this:

Make one yourself in a 3D software, or get it in the download for this demo down below!

Now, to set up the scene, add the Plane object and add a Device Camera Tracker to the camera.

You should see the purple checker board on the ground at this point.

Next up, we’re going to make an Empty Graph material and give it a Texture node. Connect the Texture node to the Shader.

If you then apply this material (I called it ‘Distortion’) to the plane, you can give the plane a texture.

Sadly, though, when you apply the Camera Device Texture to this Texture input, the camera input is not properly skewed. It looks like this:

Which makes sense – it’s just displaying the camera texture as a flat image on the plane.

We want it to be projected (‘beamed’) from the virtual camera position from where we’re looking at it.

So, every frame, the Plane should update its UVs to make the texture seem flat and aligned to the phone screen, when looked at it from the virtual camera’s perspective.

This little bit of code can do that:

//@input Component.MeshVisual plane //@input Component.Camera cam script.plane.snap(script.cam); 0 1 2 3 //@input Component.MeshVisual plane //@input Component.Camera cam script . plane . snap ( script . cam ) ;

Just put this in a Script file and place that on your Plane object.

You then have to select the Mesh Visual called ‘plane’ and the Camera called ‘cam’ for it to work, but this should not be too difficult since there is only one of them each in your scene.

Set this script to ‘Frame Updated’ and the plane will seem to disappear!

It did not actually disappear, but its texture is exactly the same as what’s behind it. So you don’t see it. If you don’t believe that, you can temporarily set the script to ‘Tapped’ and see how it actually updates its camouflage every time you tap the screen!

Now we can go ahead and distort this plane using the Shader Graph!

I’m going through distortion one more time here, but keep in mind that there are a billion different and cool kinds of effects you can make with this,

so this tutorial is giving you more of a head start than an actual final result.

Just a quick introduction to vertex displacement: there’s X, Y and Z information going in that Shader node, and all three are just numbers.

I only want the Y number to change, because Y is the up/down axis and I want that to happen.

To demonstrate what changing the Y value does, place a Surface Position node and connect it to the Shader.

Now, split it and recombine it again. This will give you control over all three axis separately.

You can now, for instance, add an Add node to the line of the Y-axis, and enter some random amount. I used 100 as the value to add.

As you can see, the entire plane is going up and it is taking its contents along with it. Those squiggly stretching lines you see are just the last known pixels to this material stretching all the way to the end of the plane, as it doesn’t know what those values are (the Device Camera Texture has reached its end at the bottom).

Get rid of the Add node, and grab a Voronoi Noise node instead! Throw the Surface Position in its Seed input, and multiply the noise by something like 100. You’ll now see these waves.

Why waves?

The world space position we’re using for the Voronoi Noise is xyz, while the seed input is only xy. This means the z information is not used for seeding the noise, and you can see that in those waves as they’re not changing in the z-axis (away/towards you)!

On the other hand, the Y axis is used, but is totally useless since the plane has the same Y value everywhere.

So we need to swap the y axis with the z axis before it goes into the Voronoi Seed input.

Grab another Combine node and wire it like this:

As you can see, I reused that Split node that was still there. No need for making a node if it already exists.

Sidenote: you can also use the Swizzle node for this. Set it to ‘xz’

This channels the z output of the Surface Position to the y input of the noise seed (and it leaves x intact)! Now, the noise will be properly wavy:

Dial down the Multiply node a bit, and add an Elapsed Time node to the Offset input of the Voronoi noise. This will make it animate, which looks way cooler.

The scaling of this noise is not right yet, though. By default the Voronoi Noise scale in the node is set to (10, 10), but because we’re working on quite a large scale I’d just go down to something like (0.018, 0.018). That looked right for me.

You can always plug the output of a node into the Color input of the Shader node to see what it looks like!

The only annoying thing left is that there are no smooth edges. Because of that, you can still see this pixel stretching at the bottom, and the end of the plane is clearly visible.

To fix that, we need to tell the Voronoi Noise to smoothly lower its output value as it reaches an edge of the plane.

We can find the edges of the plane using a Surface Position node like the one we already made earlier, except you’ll need to make a new one as its ‘Space’ setting needs to be set to ‘Object’ this time (instead of ‘World’).

Plug it into the color of your shader to preview it – this is the data it is giving us:

This is actually really useful, as it gives us a value of -1 to 1 in each direction.

Red -1 is left, red 1 is right (x-axis).

Green -1 is down, green 1 is up (y-axis).

Blue -1 is near, blue 1 is far (z-axis).

We don’t see the green one, as the plane is exactly at position 0 in height.

And the waves don’t count, because they’re calculated after this Surface Position node is getting its data.

In order to tell the noise to be at its strongest in the middle and weaker as it reaches one of the edges, we need to multiply the output of the noise by a map of the plane that has a value of 1 in the middle and a value of 0 around the edges.

Believe it or not, but we’re actually almost there. Think about it this way: the red channel is already doing a smooth ramp towards the edge on the right. It’s just doing the same but in negative values on the left.

Split this Surface Position output into three different channels, and only plug the red into the color. Now, add an ‘Abs’ node in between. That already is a ramp going from 1 to 0 to 1, indicating edges from left to right!

That ramp on the left is nothing new, it’s just the negative values inversed so they’re above 0 and thus visible (which is what the Abs node does).

Now, do the same for the z channel (which is blue), and add them both together:

Look at that! The middle is dark, towards the edges it gets brighter! Because both edges are overlapping, the values will get really bright (with values over 1).

Add another Add node to this, and set it to -1. That way, we can be sure that the maximum value will be 1.

(since, without the added -1, the maximum value of both edges combined is 1 + 1 = 2. And 2 minus 1 will get us back to 1. Quick maths.)

One last thing, the colors are inverted. White means higher values, which means more noise. And we want the most noise to be in the middle and not around the edges.

So add a ‘One Minus’ node (which is literally a node that does ‘1 – value’) to invert these colors.

Now there’s white where the noise should be, and black where it shouldn’t be.

The output of the Texture node can be plugged back into the Color input of the Shader, and the One Minus node should be multiplied by the noise output.

Add a ‘Clamp’ node between the One Minus and the Multiply node, since the One Minus will probably create lots of negative values we can now simply cut off (which is what Clamp does).

You don’t need to change any settings of the Clamp node; it’s already set to clamping everything that’s not between 0 and 1.

This is our final graph, and the noise should now only be visible in the middle of the mesh!

Download at the bottom of the page!