In my previous post I shared a brief overview of model painting and baking it back onto the texture. One problem with the initial implementation was painting through the model. That is, when painting on one side of the model, the paint will be applied to the other side of the model. This is often not desired.

painting through a model

When baking the changes in UV space, the shader will evaluate whether that portion of the screen from the camera’s view has been painted. It doesn’t have any context on which polygon has actually been painted. Only that there’s paint there. Let’s discuss a couple ways to prevent this paint through.

Strategy 1: Primitive Ids

Since the program is already drawing the polygons to a color and depth buffer (with the closest triangles in view), it can also save off the primitive id of the triangle. It can then determine if the the triangle it’s baking in texture space is the one that actually has paint on it.

primitive ids stored per pixel

When drawing the mesh in both views, the shader uses gl_PrimitiveId to record the index of the primitive being rendered. The exact number of each primitive isn’t important, but it needs to be consistent between views and render passes.

The code change for this is pretty simple. Before doing the paint lookup, first make sure that particular triangle the app is baking has been painted by doing a primitive id comparison. Otherwise use the existing color for the blend.

bool isPainted(vec3 uv)

{

highp int screenPrimitiveId = int(texture2D(drawTexture, uv.xy).a);

return screenPrimitiveId == gl_PrimitiveID;

} void main() {

...

// only apply paint color if isPainted

float paintIntensity = 0.0f;

if (isPainted(paintUv)) {

paintIntensity = texture2D(paintTexture, paintUv.xy).r;

}

...

This strategy is very simple and works pretty well, but has a pretty big problem — edges between primitives.

no more paint through, but artifacts between primitives

The primitive id lookup is turning up false negatives along the borders where the primitive id is the adjacent primitive instead. Fixing this would probably need some kind of filtering strategy.

Strategy 2: Depth Buffer

The model render pass is already saving off the color and depth of the model in the FBO’s color attachment. The bake shader is already reprojecting the fragment from UV space to camera space for the paint lookup. We can actually do a depth comparison to see if this is probably the same part of the model (and not something behind it).

Here’s the updated visibility function:

bool isPainted(vec3 uv)

{

// between 0 and 1, depth of model in FBO

float drawZ = texture2D(drawTexture, uv.xy).b; // between 0 and 1, depth of model from back projection

float meshZ = uv.z; // if the depth from the FBO and the fragment projected there

// are within an episolon, assume they're the same surface

return abs(drawZ - meshZ) < 0.0001;

}

And here’s the paint through test in action:

depth check eliminates border aliasing, but now there’s artifacting at angles

After playing with a few epsilon values for the depth comparison, the implementation works pretty well, but also introduces two new problems.

First, the bake process is now dependent on an episilon value that could need tweaking depending on the scene. Second, using a depth comparison can be very noisy especially when the primitive is at a sharp angle from the camera, where the depth variance is high between pixels in the camera view. This is the same problem rendering shadow maps, and getting shadow “acne”.

I can think of a few options to improve the comparison. One is to store a different depth value in the FBO that is farther from the first fragment in the pixel, such as the average of the first and second surfaces. Second-Depth Shadow Mapping (pdf), for example, proposes storing the second value and changing the lookup, but there are a lot of ways to improve this lookup.

Another option is to not allow the paint to be applied to surfaces not facing the camera. I’ve actually seen this trick in at least one other implementation and might actually be preferred by artists who only want to paint the most forward-facing surfaces.

Conclusion

Of the two strategies I’ve experimented with, I’m sticking with the depth check for now as it seems to be fairly robust, but may revisit the other approach or see if there’s a way to combine the two.

What’s Next

Now that paint-through is less of a problem, we can focus on seam filtering. When the textures are baked, the UV value is used to draw the polygons onto the texture. This works great for the area inside the polygon, but with texture filtering the renderer will grab samples along a “seam” that haven’t been painted, which cause rendering artifacts.

dark line along seam where samples outside of mesh on texture are sampled

When baking to the texture, we actually want to paint a bit around the edges of the mesh so the samples along the edge look correct.

Source Code

The painting code for the animations above is available on Github. It’s written in C++, OpenGL, and Qt5. It currently lacks quite a bit of documentation, but can be built in Qt Creator and assimp (for loading the 3D models).