Technique Description

Introduction

Reflection mapping is a technique used to add reflection to the surface of a 3D model by the mean of an environment map. Reflections obtained using this technique are technically not accurate but are plausible and provide a cheap and easy way to add reflections to an object compared to using raytracing (which is usually, depending on the scene's complexity, much more expensive to compute). The algorithm falls into the category of image-based lighting techniques. However the term image-based lighting was introduced quite a few years after the reflection mapping technique was invented (in the late 70s).

Technique Description

In CG, sometimes you need to integrate a 3D object to a real photograph. In the studio terminology we also call that a background plate. Photographs of real sets or places (with sometime actors in them) often represent complex environment made of potentially hundreds of objects, real people. The lighting can also be very complex (many light sources, caustics, transparencies, light bouncing off walls and objects, etc). If your 3D model is highly reflective, to properly integrate it to the plate (from a shading and lighting point of view), you would need to recreate this environment as well which means modelling many more objects, texturing them, finding their exact position in space and matching the lighting. Creating an entire 3D environment just to get reflections on a 3D model added to a background image, is a very time consuming task. Furthermore, matching the level of realism of your reference image is almost impossible. Reflections you will get from this CG environment will never be as good as the reflections you would have gotten if this object had been real and photographed in a real world environment. In the early days of CG, when CPU clocks and memory were scarce resources, lack of realism was not so much the reason why they didn't attempt at recreating these 3D environments (with a lot of manual labour they could surely have achieved it). But the rendering time and the memory required to compute these reflections was simply too expensive (you need to use raytracing and to execute complex shading operations at intersection hit points). As a workaround, artists and researchers came up with a hack known today as reflection mapping. They captured the real world environment by taking a photograph of a chrome sphere (a sphere which is 100% reflective such as silver christmas ball) placed where they were planning to add CG objects in post production.

Figure 1: photograph of a chrome ball (Uffizi Gallery in Florence, provided by Paul Debevec (c) 1999).

The good things about the chrome ball (or sometimes referred as the mirror ball or light probe as well) is that it makes it possible to capture the entire environment surrounding the sphere in one single shot (almost, we will look at the implementation details later). Because mathematics of reflections on a sphere are simple and well known it is very simple to convert this image into another format which is known as the latitude-longitude environment map (in this lesson, we will learn how to convert these images). We sometimes use the term lat-long map for short. The latitude-longitude term comes from the fact that we represent the environment surrounding the chrome sphere in a latitude-longitude format that is similar to the way earth is often represented on maps. It is just a convenient parametrisation of a sphere on a 2D plane (where horizontal lines represent positions between the two poles of the sphere, and vertical lines represents a position around the lines passing through the poles).

Figure 2: we have converted the chrome ball of the Uffizi gallery photograph into a lattitude-longitude map.

The term environment comes from the fact that this image captures the entire real-world environment surrounding the chrome ball (its environment). And if we know how to represent the surface of a sphere on a 2D plane in the form of a latitude-longitude map, we can use the reverse process to map a 3D sphere with this 2D image. This is the principle of texture mapping a 3D sphere.

Figure 3: example of a latitude-longitude map applied to a 3D sphere. The technique of mapping a sphere with such a map is simply called sphere mapping. Note the ratio of the environment map. Its width is twice its height.

The idea behind reflection mapping is to place the CG model in the centre of a very large sphere which is mapped with the latitude-longitude map we have created from the photograph of the chrome ball. Instead of raytracing a complex CG environment we simply raytrace the sphere (which is very fast) and do a lookup into the longitude-latitude map at the intersection point between the reflection ray and the sphere to get the color of the environment for that particular direction. Before we illustrate this idea, lets just give a quick reminder of what reflections are. To compute a reflection direction all you need is an incident viewing direction and the normal (N) at the point you are shading (P). The viewing direction (I) can be computed by taking the vector EP where E is the eye position. As you can see in the following figure, computing the reflection direction (R) can easily be obtained by mirroring the incident viewing direction I around the normal N. 

Figure 4: I denote the viewing incident direction. N the normal of the object at the shaded point P and R is the reflection direction. For perfect mirror reflection, the angle of incidence and the angle of reflection (θi and θr) are equal. If we surround the shaded object with a sphere, we can can find where R hits the sphere which represents the environment surrounding the 3D model. If the sphere is mapped with a latitude-longitude environment map, we can find the color in the texture at the hit point. That will become the result of the reflection at P.

In the case of perfect mirror reflections, the angle of incidence (the angle between I and N) and the angle of reflection (the angle between R and N) are the same. The equation to compute the reflection direction can then be written as:

$$R=2(N \bullet I)N-I$$

Figure 5: top: we parented this render camera to our teapot, so that when we move the teapot the camera moves with it. We used this setup to render the next images. Bottom: if the teapot moves a little bit (the camera position doesn't change in regard to the teapot, it moves with the teapot) reflections on the teapot do not seem to change as illustrated with the first two renders (in the first image the teapot is in the centre of the sphere which is also the origin. In the second image the teapot is 10 units away from the origin). That is because the sphere surrounding the object is very large. As you can see in the screen grab of the scene, the teapot (the three axises) is so small compared to the sphere that we can't even see it. However if we move the teapot very close to the sphere, the reflections change (third image). From this observation we can deduce that if make the sphere surrounding the object infinitely large, then reflections on the 3D model will be the same no matter where it is located in space.

Of course if you move the object right next to the sphere, reflections should be very different from what you get when the model is in the centre of the sphere. But because we want to come up with a generic technique that works in all possible cases and always return the same result no matter where the object is positioned in 3D space, we will assume that in fact, the sphere that is surrounding the model is infinitely large. Raytracing an infinitely large sphere doesn't seem maybe the best of possible thing to do, but in fact, literally adding a sphere to your scene and raytracing it, is not necessary in that case anyway. Once again mathematics will be used to solve this problem analytically. Because the sphere is infinitely large, you can make the assumption that the location of the 3D model does not affect the result of the reflections. In other words you can assume that your object is right in the centre of the sphere. In that particular case, all the reflection directions that are computed for the object, can be seen as rays originating from the centre of the sphere as well (if the sphere in infinitely large, the object can be seen as infinitely small and therefore the origin of the reflection rays becomes negligible in solving the problem). This is particularly useful because it reduces the reflections rays to vectors centred at the origin and pointing to specific directions over the unit sphere (if the directions are normalised). There is no need to compute intersections between the sphere and the reflection rays. All that is needed is to convert reflection vectors into sets of 2D coordinates (uv coordinates) that correspond to pixel positions in the latitude-longitude map (as illustrated in the following figure).

// Figure 6 insert figure of unit refl vector + mapped sphere with UV coordinate

To get reflection mapping there is not need to add a sphere to the scene. The reflection directions can be computed using the equation given above and they can be converted (we will look at the math in chapter 2) into uv coordinates using simple arithmetics. Once we have these coordinates we can do a lookup (reading the pixel color in the texture for that uv coordinate) in the latitude-longitude map and set the reflection color with the pixel color returned from that lookup. Simple.

You might ask, what's difficult about reflection mapping then ? Nothing is really hard about it. However they are a few subtleties about the technique and having a reference document like this lesson is convenient to remember them all. Usually mathematics for converting vectors to uv coordinates (and vice and versa) are pretty straightforward but not something you want to learn by heart. If you know the basic of trigonometry, you should be able to find these equations by yourself easily. You might also need a tool to convert various probes (chrome balls) into latitude-longitude maps. It is worth mentioning now that the latitude-longitude map is not the only format that is used for reflection mapping. Latitude-longitude maps are simple but they are not ideal (we will explain why in the next chapter). Other formats have been develop to address these problems. The most popular ones are cube maps, angular maps, and paraboloid maps (both are explained in this lesson). Note also that the mirror ball can be used directly (without converting it into a lat-long image). You can directly convert a reflection direction into a uv coordinates valid for making a lookup in the mirror ball image. We generally use lat-long images because they make it easier to visualise the environment (compared to a mirror ball image) and they can be painted (touched up) in a 2D program more easily as well.

Figure 7: an environment map can be seen as a way of recording the incident light illumination.

It is also convenient to know how to convert a latitude-longitude map back into a mirror ball or any of the formats we just mentioned. Converting these images is usually simple but we will need to take care of filtering a little to get the best possible results. It's worth saying that nowadays most chrome ball images are HDR images (High Dynamic Range images). They are usually in HDR file formats such as OpenEXR or HDR (Radiance file format). Knowing how to read these images can be particularly useful. It's particular important to remember that what these chrome balls represent is not only the environment surrounding the object but also the incident light illumination. In other words a chrome ball is light and it can be used as a light source to illuminate CG models. This is the foundation of image-based lighting. Reflection mapping falls in the category of image-based lighting techniques.

Finally, it's worth noticing that it is possible to use reflection mapping to get cheap blurred reflections. Instead of sampling the material, we can simply apply a blur to the environment map. This technique will be explained in the next lesson.

Figure 8: by blurring the environment map which is fast (if we use mip mapping for instance), it is possible to easily change change the roughness of the reflections.

History

It's actually quite hard to know the exact history of the technique. If you have some information on this topic please contact us. However it seems that in 1974, Catmull was the first person to describe a technique using a texture to simulate reflections in a paper entitled "Computer Display of Curved Surfaces". This work would have been developed later on (1976) by Blinn and Newell ("Texture and Reflection in Computer Generated Images"). MAGI would have been the first production renderer implementing this technique (MAGI was used on Tron). Here is an image that was rendered with this 3D program. We do not own the rights on this image and do not have proper credits for it. We would like to get in touch with someone who could give us more information about this image (who made it, when was it made, what was the context). Please use the contact form to get in touch with us.

The technique was made much more popular when Terminator 2 was released in 1991. ILM used it quite extensively to developed the look of the character T-1000 when he was in his liquid state. These shots were revolutionary at the time.

Chapter 1 of 2 Next Chapter »