OpenGL supports an object selection mechanism in which
the object geometry is transformed and compared against
a selection subregion of a window. The mechanism uses the
transformation pipeline to compare object vertices against
the view volume. To reduce the view volume to a screen-space
subregion (window coordinates), the projected coordinates of
the object are transformed by the following matrix
Objects are identified by assigning them integer names using glLoadName(). As each object is sent to the OpenGL pipeline and tested against the pick region, if the test succeeds a hit record is created to identify the object. The hit record is written to the selection buffer whenever a change is made to the current object name. An application can determine which objects intersected the pick region by scanning the selection buffer and examining the names present in the buffer.
The OpenGL selection method determines that an object has been hit if it intersects the view volume. Bitmap and pixel image primitives will generate a hit record only if a raster positioning command is sent to the pipeline and the transformed position lies within the viewing volume. To generate hit records for any point within a pixel image or bitmap, a bounding rectangle should be sent rather than the image, and selection testing performed on the interior of the rectangle. Similarly, wide lines and points will be selected only if the equivalent infinitely thin line or small point would be selected. To facilitate selection testing of wide lines and points, proxy geometry representing the true footprint of the primitive should be used instead.
Many applications take advantage of instancing of geometric data to reduce the memory footprint. Instancing allows an application to create a single representation of the geometric data for each type of object used in the scene. If the application is modeling a car, the four wheels of the car may be represented as instances of a single geometric description of a wheel combined with a modeling transformation to place the wheel in the correct location in the scene. Instancing introduces extra complexity into the picking operation. If a single name is associated with the wheel geometry, the application can not determine which of the four instances of the wheel has been picked. OpenGL solves this problem by maintaining a stack of object names. This allows an application that represents a model hierarchically to associate a name at each stage in the hierarchy. As the hierarchical model is drawn new names are pushed onto the stack as the hierarchy is descended and old names are popped as the hierarchy is ascended. When a hit record is created, it contains all of the names currently in the name stack. The application can determine which instance of an object is selected by looking at the contents of the name stack and comparing them to the names stored in the hierarchical representation of the model.
Using the car model example, the application associates an object name with the wheel representation and another object name with each of the transformations used to position the wheel in the car model. The application determines that a wheel is selected if the selection buffer contains the object name for the wheel, and it determines which instance of the wheel by examining the object name of the transformation.
When the OpenGL pipeline is in selection mode, the primitives sent to the pipeline do not generate fragments to the framebuffer. Since only the result of the transformation pipeline is of interest, there is no need to send texture coordinates, normals, etc. or to enable lighting.