8.7 Compositing Images with Depth

Compositing separate images together is a useful technique for increasing the complexity of a scene [27]. Section 8.1 discusses algorithms for compositing two images together using alpha values to control how pixels are merged. One drawback of this method is that only simple visibility information can be expressed using mattes or masks. Using the stencil buffer, it is possible to merge images using the original depth information from the images. Both color and depth images can be independently saved to memory and later drawn to the screen using glDrawPixels(). This is sufficient for 2D style composites, where objects are drawn on top of each other to create the final scene. To do true 3D compositing, it is necessary to use the color and depth values simultaneously, so that depth testing can be used to determine which surfaces are obscured by others.

The stencil buffer can be used for true 3D compositing in a two pass operation. The color buffer is disabled for writing, the stencil buffer is cleared, and the saved depth values are copied into the framebuffer. Depth testing is enabled, insuring that only depth values that are closer to the original can update the depth buffer. glStencilOp() is called to set a stencil buffer bit if the depth test passes.

The stencil buffer now contains a mask of pixels that were closer to the view than the pixels of the original image. The stencil function is changed to accomplish this masking operation, the color buffer is enabled for writing, and the color values of the saved image are drawn to the frame buffer.

This technique works because the fragment operations, in particular the depth test and the stencil test, are part of both the geometry and imaging pipelines in OpenGL. Here is the technique in more detail. It assumes that both the depth and color values of an image have been saved to system memory, and are to be composited using depth testing to an image in the framebuffer:

1. Clear the stencil buffer using glClear(), or'ing in GL_ STENCIL_BUFFER_BIT.
2. Disable the color buffer for writing with glColorMask().
3. Set stencil values to 1 when the depth test passes by calling glStencilFuncGL_ ALWAYS, 1, 1(GL_ ALWAYS, 1, 1), and glStencilOpGL_ KEEP, GL_ KEEP, GL_ REPLACE(GL_ KEEP, GL_ KEEP, GL_ REPLACE).
4. Ensure depth testing is set; glEnableGL_ DEPTH_TEST(GL_ DEPTH_TEST), glDepthFuncGL_ LESS(GL_ LESS).
5. Draw the depth values to the framebuffer with glDrawPixels(), using GL_ DEPTH_COMPONENT for the format argument.
6. Set the stencil buffer to test for stencil values of 1 with glStencilFuncGL_ EQUAL, 1, 1(GL_ EQUAL, 1, 1) and glStencilOpGL_ KEEP, GL_ KEEP, GL_ KEEP(GL_ KEEP, GL_ KEEP, GL_ KEEP).
7. Disable the depth testing with glDisableGL_ DEPTH_TEST(GL_ DEPTH_TEST).
8. Draw the color values to the framebuffer with glDrawPixels(), using GL_ RGBA as the format argument.

At this point, both the depth and color values will have been merged, using the depth test to control which pixels from the saved image would update the framebuffer. Compositing can still be problematic when merging images with coplanar polygons.

This process can be repeated to merge multiple images. The depth values of the saved image can be manipulated by changing the values of GL_ DEPTH_SCALE and GL_ DEPTH_BIAS with glPixelTransfer(). This technique could allow you to squeeze the incoming image into a limited range of depth values within the scene.