Textures, Filtering and Mipmaps

When writing simple applications using OpenGL, it’s not uncommon to use a library such as stb_image or SOIL to load a PNG or JPEG image as a texture. This is nice and convenient. The ease of use doesn’t come with downsides, however. The compression used for JPEG and PNG leaves the image in a format unsuitable for the random-access required by texture sampling on the GPU. To make these images usable, they are decompressed before being uploaded. The decompression step, while quick, can add up if you have a large number of textures or a very tight budget. For example if you are streaming data in-game, naïvely decompressing many JPGs or PNGs will eat into your 16.6ms budget.

Unless you are rendering a screen-sized texture onto a screen-sized quad, each pixel in the image is unlikely to cover 1 pixel on the screen. To improve the quality of the texture when viewed smaller, larger, and at shallow angles, filtering techniques are used to get a plausible colour for each sample. Mipmaps are the most important for this post.

The idea behind mipmaps is that rather than have a single screen pixel cover 10s or more texture pixels, a smaller version of the texture is used that has better coverage, requiring fewer samples. Thus each texture is not an individual image, but a series of mipmap levels, each half the size of the previous level. To give an example, a texture that’s 256x256 pixels, could have mipmaps for 128x128, 64x64, 32x32, and so on. If the view was facing directly at a plane with this texture (let’s say it’s a brick wall), then depending on the distance and viewport size moving away from the wall would select progressively smaller mipmaps for use to keep the ratio of texture pixels to screen pixels reasonable.

Usual image formats don’t support storing these mipmap chains. When loading a JPG or a PNG, mipmaps must be generated by resizing the original image repeatedly for each required mipmap level. That’s no problem if you only have 10 textures, but if you have 100s of textures or need to stream in textures at run-time, then spending time generating these smaller images will be something worth reducing.

Texture containers are a solution to these problems. Not only do they store texture data suitable for direct use as textures, they also store mip-map and array data. This means a single texture container can store all the data required for an entire texture, no matter it’s size, number of mipmaps, array layers or cubemap faces. Generating mipmaps offline is important if you use compressed textures, as it’s generally impractical to generate compressed textures at runtime.

Container formats

The most common texture container formats in use that store data compatible with D3D, OpenGL or Vulkan, are the Khronos Texture format (KTX) and DirectDraw Surface (DDS).

Khronos Texture (KTX)

Khronos Texture is a Khronos Group standard for storing textures intended for use with OpenGL and OpenGL ES. The specification for the format is available via Khronos. It contains format information directly compatible with OpenGL and OpenGL ES enumerations. Also included is the ability to store application specific key-value data within the file.

The specification provides a good overview of the format and the expected values for each of the header fields. The format can store 1D, 2D, 3D, Cubemaps and Array Textures, along with any number of mipmaps for these textures. This makes it ideal for storing almost any kind of texture you could want.

The texture data is described in theglType, glFormat, glInternalFormat, and glBaseInternalFormat header fields. These should match up with the parameters to the gl[Compressed]Tex[Sub]Image*D calls used to submit each texture mipmap level’s data.

DirectDraw Surface (DDS)

Despite DirectDraw being long deprecated, the DDS format is still in common use for storing textures used with Direct3D. Originally only 2D textures were supported, but the D3D10 header extension added support for texture arrays and D3D10+ features. The format is partially documented along with expected values on MSDN, useful for integrating DDS into your pipeline.

For loading DDS in D3D11, there is a deprecated function, D3DX11CreateTextureFromFile, which is no longer available in Windows Store apps. DirectXTK has a replacement, DDSTextureLoader that can be used instead.

Use containers

Since KTX is somewhat better specified that DDS, I would recommend using it over DDS. Howerver, as long as you use a sensible format that simplifies storing and loading, you should benefit from the increased simplicity and reduced loading times. There is an official KTX library libktx for working with KTX files. More information about this is available on the Khronos website.