As part of our day-to-day at NVIDIA we’ve been helping a developer to integrate one of our GameWorks libraries into the Unity engine. The GameWorks library in question is HBAO+ and after a quick initial assessment we discovered that it would be possible to achieve this without; A) needing to modify the internals of the library itself and B) use the existing functionality available in Unity for creating graphics plugins. Both of these are great news, as it means the process is accessible to any user of Unity.

HBAO+

HBAO+ (stands for Horizon Based Ambient Occlusion Plus) is part of the GameWorks family of visual effects libraries. Its purpose is to provide a fast, hardware vendor invariant (that is to say; it works on all GPU’s, not just NVIDIA’s) way of rendering ambient occlusion in games. It’s been successfully integrated into a vast number of games, most recently including: - “Assassin’s Creed Unity”, and “Far Cry 4”.

You Will Need

Before getting started with integrating a GameWorks library into Unity, there are a couple things you’ll need;

Firstly, you’re going to need a full license for Unity. As it stands, in order to make use of the native plugin feature required for the integration, this will need to be a “PRO” Unity license.

TIP: If you haven’t created a “Low-level Native Plugin Interface” for Unity3D before, a comprehensive introduction for doing this can be found on the Unity3D documentation site here

Secondly, a GameWorks library of your choosing will be necessary. For the purpose of this article, we’re going to use HBAO+ as an example, as this is publicly available (subject to a EULA) from our ShadowWorks page here.

The HBAO+ library is a self-contained library, in the form of a DLL (at least on Windows). Along with the HBAO+ DLL, you should expect to get a C++ header file which contains the interface for HBAO+.

Design

We decided to create an intermediate plugin interface to facilitate requests from a Unity script to the HBAO+ library. Using this design we keep a very simple and clean API within Unity itself, and manage all the lower level functionality required for the HBAO+ library in our C++ low level plugin, where we have access to the DirectX API.



Interaction between Unity, HBAO+ and the plugin.

Initialisation

Initialisation of the HBAO+ library needs to be handled once, this can be done when Unity calls into a low level plugin to initialise a device pointer. This is done by implementing the following function in the low level plugin:

extern "C" void UnitySetGraphicsDevice (void* device, int deviceType, int eventType);

With this function implemented, Unity can now make a call into the plugin which allows it to get a graphics device and context; it’s at this time that the plugin can now initialise the HBAO+ library using the method, GFSDK_SSAO_CreateContext_XXX, where XXX is the name of the graphics API (e.g. D3D11 or GL). This method is called automatically by the engine and nothing needs to be implemented in a Unity script for this to happen.

Rendering

To render HBAO+, a call must be made to GFSDK_SSAO_RenderAO_XXX (again XXX…). In order to make this call from Unity, we must implement it in our plugin, and then tell Unity when we would like the call to happen, this is done via a rendering event system. It’s worth noting that Unity implements a multithreaded rendering system, which is transparent to the average Unity scripter. It’s for this reason that we can’t just blindly make a graphics call to our plugin from a Unity script.

Luckily Unity has a system in place for helping out in circumstances like these, an event based system, which enables a Unity script to raise an event at the point where the rendering code should occur, and Unity will handle the multithreaded untangling in the backend. All that needs to be done in the Unity script is use the following method to raise a rendering event:

public static void GL.IssueRenderEvent(int eventID);

And in the plugin, the following external method must be implemented to catch the events:

extern "C" void UnityRenderEvent (int eventID);

The value of the eventID parameter is defined when raising the event, in this case our plugin will just be handling a single eventID, but we must account for multiple eventID’s being called as we don’t know how many other low level plugins the client developer might be using. Once the check for eventID has passed, now the plugin can make the render call into the HBAO+ library.

TIP: In order to share the correct eventID between the plugin and Unity script, a simple getter method can be implemented within the plugin, and then called from Unity script.

When the Going is Tough!

Being a screenspace ambient occlusion algorithm, HBAO+ requires (at the very least) a copy of the depth texture. This is tricky when using Unity as users aren’t exposed to the depth texture as a tangible resource. The only exposure to the depth texture comes in the form of the built-in shader variable, ‘_CameraDepthTexture’. In the sample included with this post you’ll see that the Unity project contains a shader called, ‘FetchDepth’. This shader declares the built-in variable for the depth texture with an explicit register binding which will force Unity to use this register when binding the SRV. We use this to our advantage by calling Material.SetPass(…) on a material created using this dummy shader right before we raise the render event for our plugin. Since we know the register for which ‘_CameraDepthTexture’ is bound, this allows the plugin to use the DirectX 11 API for fetching bound resources (ID3D11DeviceContext::PSGetShaderResources) in order to get a pointer to the SRV for the depth texture.

Unity wraps up a texture device resource with the class “RenderTexture”. When passing this RenderTexture to the C++ plugin, it must be done by sharing the resource pointer (which can be found using the RenderTexture’s member method, “GetNativePtr()”). However, this means that only the resource itself will be shared between the Unity and the plugin, and the various views required (SRV for depth texture and RTV for output buffer) will have to be created and cached by the plugin.

Unity provides no event callback for when the window has been resized. This is a problem for us as we will need to resize our staging depth buffer and output buffer. In order to work around this, the Unity script simply checks the current resolution of the staging buffers against the current screen resolution and in the event of a change, resizes the staging buffers.

TIP: When writing an image effect in Unity, it’s common to use the “OnImageRender” callback to handle the rendering logic. When implementing an SSAO algorithm, it’s usual to perform the AO immediately after opaque rendering (before blended). In Unity script, the OnImageRender callback can be made to fire at this exact time by using the “ImageEffectOpaque” attribute.

Finishing

Here we’ve shown that it’s possible for the advanced Unity user to integrate a GameWorks library into the Unity engine. Hopefully this article will help anyone trying to do a similar thing with other GameWorks libraries.