This is the eleventh part of a tutorial series about rendering. Previously, we made our shader capable of rendering complex materials. But these materials have always been fully opaque. Now we'll add support for transparency.

This tutorial was made with Unity 5.5.0f3.

Cutout Rendering

To create a transparent material, we have to know the transparency of each fragment. This information is most often stored in the alpha channel of colors. In our case, that's the alpha channel of the main albedo texture, and the alpha channel of the color tint.

Here is an example transparency map. It's a solid white texture with fading smooth noise in the alpha channel. It's white so we can fully focus on the transparency, without being distracted by an albedo pattern.

Transparency map on a black background.

Assigning this texture to our material just makes it white. The alpha channel is ignored, unless you chose to use it as the smoothness source. But when you select a quad with this material, you'll see a mostly-circular selection outline.

Selection outline on a solid quad.

How do I get a selection outline? Unity 5.5 introduced a new selection highlighting method. Previously, you always saw a wireframe of the selected mesh. Now you can also choose to use an outline effect, via the Gizmos menu of the scene view. Unity creates the outline with a replacement shader, which we'll mention later. It samples the main texture's alpha channel. The outline is drawn where the alpha value becomes zero.

Determing the Alpha Value To retrieve the alpha value, we can use add a GetAlpha function to the My Lighting include file. Like albedo, we find it by multiplying the tint and main texture alpha values. float GetAlpha (Interpolators i) { return _Tint.a * tex2D(_MainTex, i.uv.xy).a; } However, we should only use the texture when we're not using its alpha channel to determine the smoothness. If we didn't check for that, we could be misinterpreting the data. float GetAlpha (Interpolators i) { float alpha = _Tint.a; #if !defined(_SMOOTHNESS_ALBEDO) alpha *= tex2D(_MainTex, i.uv.xy).a ; #endif return alpha; }

Cutting Holes In the case of opaque materials, every fragment that passes its depth test is rendered. All fragments are fully opaque and write to the depth buffer. Transparency complicates this. The simplest way to do transparency is to keep it binary. Either a fragment is fully opaque, or it's fully transparent. If it is transparent, then it's simply not rendered at all. This makes it possible to cut holes in surfaces. To abort rendering a fragment, we can use the clip function. If the argument of this function is negative, then the fragment will be discarded. The GPU won't blend its color, and it won't write to the depth buffer. If that happens, we don't need to worry about all the other material properties. So it's most efficient to clip as early as possible. In our case, that's at the beginning of the MyFragmentProgram function. We'll use the alpha value to determine whether we should clip or not. As alpha lies somewhere in between zero and one, we'll have to subtract something to make it negative. By subtracting ½, we'll make the bottom half of the alpha range negative. This means that fragments with an alpha value of at least ½ will be rendered, while all others will be clipped. float4 MyFragmentProgram (Interpolators i) : SV_TARGET { float alpha = GetAlpha(i); clip(alpha - 0.5); … } Clipping everything below alpha 0.5.

Variable Cutoff Subtracting ½ from alpha is arbitrary. We could've subtracted another number instead. If we subtract a higher value from alpha, then a large range will be clipped. So this value acts as a cutoff threshold. Let's make it variable. First, add an Alpha Cutoff property to our shader. Properties { … _AlphaCutoff ("Alpha Cutoff", Range(0, 1)) = 0.5 } Then add the corresponding variable to My Lighting and subtract it from the alpha value before clipping, instead of ½. float _AlphaCutoff; … float4 MyFragmentProgram (Interpolators i) : SV_TARGET { float alpha = GetAlpha(i); clip(alpha - _AlphaCutoff ); … } Finally, we also have to add the cutoff to our custom shader UI. The standard shader shows the cutoff below the albedo line, so we'll do that as well. We'll show an indented slider, just like we do for Smoothness. void DoMain () { GUILayout.Label("Main Maps", EditorStyles.boldLabel); MaterialProperty mainTex = FindProperty("_MainTex"); editor.TexturePropertySingleLine( MakeLabel(mainTex, "Albedo (RGB)"), mainTex, FindProperty("_Tint") ); DoAlphaCutoff(); … } void DoAlphaCutoff () { MaterialProperty slider = FindProperty("_AlphaCutoff"); EditorGUI.indentLevel += 2; editor.ShaderProperty(slider, MakeLabel(slider)); EditorGUI.indentLevel -= 2; } Alpha cutoff slider. Now you can adjust the cutoff as you like. You could also animate it, for example to create a materializing or de-materializing effect. Varying alpha cutoff. The shader compiler converts a clip to a discard instruction. Here's the relevant OpenGL Core code fragment. u_xlat10_0 = texture(_MainTex, vs_TEXCOORD0.xy); u_xlat1.xyz = u_xlat10_0.xyz * _Tint.xyz; u_xlat30 = _Tint.w * u_xlat10_0.w + (-_AlphaCutoff); u_xlatb30 = u_xlat30<0.0; if((int(u_xlatb30) * int(0xffffffffu))!=0){discard;} And here it is for Direct3D 11. 0: sample r0.xyzw, v1.xyxx, t0.xyzw, s1 1: mul r1.xyz, r0.xyzx, cb0[4].xyzx 2: mad r0.w, cb0[4].w, r0.w, -cb0[9].x 3: lt r0.w, r0.w, l(0.000000) 4: discard_nz r0.w What about shadows? We'll take care of shadows for cutout and semitransparent materials in the next tutorial. Until then, you can turn off shadows for objects using those materials.

Rendering Mode Clipping doesn't come for free. It isn't that bad for desktop GPUs, but mobile GPUs that use tiled rendering don't like to discard fragments at all. So we should only include the clip statement if we're really rendering a cutout material. Fully opaque materials don't need it. To do this, let's make it dependent on a new keyword, _RENDERING_CUTOUT. float alpha = GetAlpha(i); #if defined(_RENDERING_CUTOUT) clip(alpha - _AlphaCutoff); #endif Add a shader feature for this keyword, both to the base pass and the additive pass. #pragma shader_feature _RENDERING_CUTOUT #pragma shader_feature _METALLIC_MAP In our custom UI script, add a RenderingMode enumeration, offering a choice between opaque and cutout rendering. enum RenderingMode { Opaque, Cutout } Add a separate method to display a line for the rendering mode. We'll use an enumeration popup based on the keyword, like we do for the smoothness source. Set the mode based on the existence of the _RENDERING_CUTOUT keyword. Show the popup, and if the user changes it, set the keyword again. void DoRenderingMode () { RenderingMode mode = RenderingMode.Opaque; if (IsKeywordEnabled("_RENDERING_CUTOUT")) { mode = RenderingMode.Cutout; } EditorGUI.BeginChangeCheck(); mode = (RenderingMode)EditorGUILayout.EnumPopup( MakeLabel("Rendering Mode"), mode ); if (EditorGUI.EndChangeCheck()) { RecordAction("Rendering Mode"); SetKeyword("_RENDERING_CUTOUT", mode == RenderingMode.Cutout); } } Like the standard shader, we'll show the rendering mode at the top of our UI. public override void OnGUI ( MaterialEditor editor, MaterialProperty[] properties ) { this.target = editor.target as Material; this.editor = editor; this.properties = properties; DoRenderingMode(); DoMain(); DoSecondary(); } Rendering mode choice. We can now switch between fully opaque and cutout rendering. However, the alpha cutoff slider remains visible, even in opaque mode. Ideally, it should only be shown when needed. The standard shader does this as well. To communicate this between DoRenderingMode and DoMain , add a boolean field that indicated whether the alpha cutoff should be shown. bool shouldShowAlphaCutoff; … void DoRenderingMode () { RenderingMode mode = RenderingMode.Opaque; shouldShowAlphaCutoff = false; if (IsKeywordEnabled("_RENDERING_CUTOUT")) { mode = RenderingMode.Cutout; shouldShowAlphaCutoff = true; } … } void DoMain () { … if (shouldShowAlphaCutoff) { DoAlphaCutoff(); } … }

Rendering Queue Although our rendering modes are now fully functional, there is another thing that Unity's shaders do. They put cutout materials in a different render queue that opaque materials. Opaque things are rendered first, followed by the cutout stuff. This is done because clipping is more expensive. Rendering opaque objects first means that we'll never render cutout objects that end up behind solid objects. Internally, each object has a number that corresponds with its queue. The default queue is 2000. The cutout queue is 2450. Lower queues are rendered first. You can set the queue of a shader pass using the Queue tag. You can use the queue names, and also add an offset for more precise control over when objects get rendered. For example, "Queue" = "Geometry+1" But we don't have a fixed queue. It depends on the rendering mode. So instead of using the tag, we'll have our UI set a custom render queue, which overrules the shader's queue. You can find out what the custom render queue of a material is by selecting it while the inspector is in debug mode. You'll be able to see its Custom Render Queue field. Its default value is −1, which indicates that there is no custom value set, so the shader's Queue tag should be used. Custom render queue. We don't really care what the exact number of a queue is. They might even change in future Unity versions. Fortunately, the UnityEngine.Rendering namespace contains the RenderQueue enum, which contains the correct values. So let's use that namespace in our UI script. using UnityEngine; using UnityEngine.Rendering; using UnityEditor; public class MyLightingShaderGUI : ShaderGUI { … } When a change is detected inside DoRenderingMode , determine the correct render queue. Then, iterate through the selected materials and update their queue overrides. if (EditorGUI.EndChangeCheck()) { RecordAction("Rendering Mode"); SetKeyword("_RENDERING_CUTOUT", mode == RenderingMode.Cutout); RenderQueue queue = mode == RenderingMode.Opaque ? RenderQueue.Geometry : RenderQueue.AlphaTest; foreach (Material m in editor.targets) { m.renderQueue = (int)queue; } }