Real World Cameras

Most of the techniques we will be studying from that lesson onwards, will use what we have learned so far about colors, points, vectors, matrices and trigonometry. Be sure you have covered these grounds first before you start reading this chapter. The ultimate goal of a renderer is to produce an image which is either stored on disk (in the form of an image file) or displayed on the screen. You should also be familiar with the idea that a digital image, is a two dimensional array of pixels and that pixels encode colors using an additive RGB color model (as explained in the lesson 5).

Camera Obscura: How is an Image Formed?

Most algorithms we use in computer graphics simulate how things work in the real world. This is particularly true of virtual cameras which are fundamental to the process of creating a computer graphics image. The creation of an image in a real-world camera is actually pretty simple to reproduce with a computer. It mainly relies on simulating the way light travels in space, interacts with objects and is deformed by optical systems such as lenses. The light-matter interaction process is highly complex but the laws of optics are relatively simple and can easily be simulated in a computer program. There is two main part to the principle of photography: the process by which an image is stored on film or to a file and the process by which this image is actually created in the camera. Film was only necessary because we actually had no better way until recently (with the introduction of digital technology and sensors) to store an image on a physical media. Most of us have developed a sentimental attachment to film however like most techniques involving chemical processes, film is not the best way of storing and making images because to say things quickly, chemicals are unstable thus hard to control. Digital technology (we won't argue about the artistic side of things which is subjective) doesn't have this disadvantage: it is predictable and therefore much more reliable than film. Taking the same photograph of the same subject under the same lighting conditions, is guaranteed to produce the same image (which is not the case with film). In computer graphics, we don't need a physical support to store an image thus simulating the photochemical processes used in traditional film photography won't be necessary (unless like the Maxwell renderer, you want to provide a realistic camera model but this is not necessary to get a basic model working).

Figure 1: The pinhole camera and camera obscura principle illustrated in 1925, in The Boy Scientist.

Figure 2: a camera obscura is a box with a hole on one side. Light passing through that hole forms an inverted image of the scene on the opposite side of the box.

Now lets talk about the second part of the photography process: how images are formed in the camera. The principle of the image creation process is actually very simple and showed in the reproduction of this illustration published in the early 20th century (figure 1. We have added colors to this old figure to make the demonstration easier). In the setup from figure 1, the first surface (in red) blocks light from reaching the second surface (in green). However if you make a small hole (a pinhole), light can traverse the first surface in one point, and light rays passing through that point will form an (inverted) image of the candle on the other side (if you follow the path of these rays you can see how this image is geometrically constructed). In reality, the image of the candle will be very hard to see because the amount of light emitted by the candle actually passing through point B is really very small compared to the overall amount of light emitted by the candle itself (trying to see the effect of a flashlight directed towards objects on a bright day light is a similar problem). To block light coming out of the candle from reaching our eyes, we can put ourselves in a box and make a small hole on one of its sides. No light will get inside this box apart from a small amount passing through the pinhole. Looking at the face opposite the hole you should now see the inverted image of the candle much more easily (figure 2). This is the principle of the camera obscura (literally it means dark room) which is the ancestor of the camera. You can perfectly turn your own room into a camera obscura and this video from the National Geographic (all rights reserved) shows how:

To perceive the projected image on the wall your eyes first need to adjust to the darkness of the room, and to capture the effect on a camera, long exposure times are needed (about 30 seconds in the video from the National Geographic). To turn your camera obscura into a pinhole camera all you need to do is put a piece of film on the face opposite the pinhole. If you wait long enough (and keep the camera perfectly still), light will modify the chemicals on the film and a latent image will be formed. The principle for digital camera is the same; the film is replaced by a sensor that converts light into electrical charges.

How Does Real Camera Work?

In real camera, images are created when light falls on a surface which is sensitive to light (note that this actually also true for the eye). For a film camera, this is the surface of the film and for a digital camera this is the surface of a sensor (or CCD). Some of these concepts have been explained in lesson 1 but we will explain them again here briefly. In the real world, light comes from various sources (the most important one being the sun).

Figure 3: in the real world, when the light from a light source reaches an object, it is reflected back into the scene in many directions. However only one ray goes in the direction of the camera and hits the surface of the film or CCD.

When light hits an object, it can either be absorbed or reflected back into the scene. This phenomenon will be explained in details in the lesson on light-matter interaction. When you take a picture, some of that reflected light (in the form of packets of photons) will travel in the direction of the camera (or the eye), pass through the pinhole, and an image of the scene will be captured on the film or by the sensor. We have illustrated this process in figure 3.

Many documents on how photographic film works can be found on the internet. Lets just mention that a film which is exposed to light doesn't generally directly create a visible image. It produces what we call a latent image (invisible to the eye) and we need to process the film with some chemicals in a darkroom to make it visible.

We were able to break apart a disposable camera to show you how things work in a real camera (image below). This camera uses a lens but the principles by which an image is formed on the back on the camera (normally on the surface of the film), are the same (check the lesson in the advanced section on Cameras: Advanced Techniques to learn more about lenses. We also breifly talk about lenses further down in this chapter). We have positionned a translucent plastic sheet where the film normally stands, to see the image which is normally formed there (and imprinted on the film). Note the inverted image (top left. Top right is a close up of the image itself).

Pinhole Cameras

The simplest type of camera we can find in the real world is the pinhole camera. It's a simple light-proof box with a very small hole in the front which is also called an aperture, and some light-sensitive film paper laid inside the box on the side facing this pinhole. The principle is very simple. When you want to take a picture you simply open the aperture to expose the film to light (to prevent light from entering the box, you keep a piece of opaque tape on the pinhole which you remove to take the photograph and put back afterwards). The principle of the pinhole camera which have already explained is simple. Objects from the scene reflect light in all directions (you will learn more about this topic in the lesson on light-matter interaction).

Figure 4: principle of a pinhole camera. Light rays (which we have artificially colored to better track their path) converge at the aperture and form an inverted image of the scene at the back of the camera, on the film plane.

The size of the aperture is so small that only one ray among the many that are reflected at this point can enter the camera (in reality it's never exactly one ray, but more a bundle of light rays or photons composing a very narrow light beam). In figure 3, we can see how one single light ray among the many reflected at P passes through the aperture. In figure 4, we have colored six of these rays to track their path to the film plane more easily; notice one more time by following these rays how they form an inverted image of the subject. In geometry, the pinhole is also called the center of projection; all rays entering the camera converge to this point and diverge after passing through it. To summarize what we have learned so far, light striking an object is reflected back in random directions in the scene (actually all directions are contained within a hemisphere centred around the point of reflection and oriented along the surface normal at that point) but only one of these rays (or a bundle of these rays traveling along the same direction) passes through the pinhole to strike the film in one single point. It is important to notice that a small area on the surface on the object (which for simplification we call a point) corresponds to a single small area on the surface of the image (which we also call a point), the footprint of the light beam striking the film surface. If you now apply what we have described for one single point on the surface to the infinity of points making up the surface of the objects form the scene, for each on of these points you have a ray passing through the pinhole and striking the back face of the camera. When all these rays strike the surface of the film, they form an image. You can visualize this process by tracing lines from an infinity of points on the surface of all the objects in the scene through the aperture and mark the location where these rays strike the film with a colored dot. The color of each one of these points is simply the color of the object at the origin of the ray (to better understand what is the color of objects, check the lesson on light-matter interaction). In lesson 1 we have mentioned that this process is called forward tracing (because we follow the natural path of light from the light source to the object and from the object to the film). In this demonstration we have used the concept of point to describe what's happening locally at the surface of an object (and what's happening locally at the surface of the film), however the surface of objects is continuous (at least at the macroscopic level) therefore the image of these objects on the surface of the film also appears as continuous.

At the microscopic level (small) we can't say that the material world is continuous because as we know it today, matter is actually composed of individual small elements (atoms and molecules). What we see and experience of the world at the macroscopic scale (large) is actually the result of an extremely large number of the small particles (atoms, molecules) interacting with each other. However without getting into too much mathematical details here, the number of particles contained in a macroscopic object (pretty much every object you can see around you) is so large, that the mathematical models we use to describe physical phenomenon observed at that scale, treat matter as a continuum rather than as particles (if you are interested in this topic, check the lesson on light-matter interaction).

Figure 5: top, when the pinhole is small only a small set of rays are entering the camera. Bottom, when the pinhole is much larger, the same point from an object, appears multiple time on the film plane. The resulting image is blurred.

Figure 6: in reality, light rays passing through the pinhole can be seen as forming a small cone of light. Its size depends on the diameter of the pinhole (top). When the cones are too large, the disk of light they project on the film surface overlap, which is the cause of blur in images.

The size of the aperture matters. To get a fairly sharp image each point (or small area) on the surface of an object needs to be represented as one single point (a small area, the footprint of the light beam) on the film. As we mentioned before it's never exactly one ray which is passing through the pinhole but more a small set of rays contained within a cone of directions (which size is defined by the size of the pinhole. Figure 6).

Figure 7: the smaller the pinhole the sharper the image. When the aperture is too large, the image is blurred.

Figure 8: circles of confusion are much more visible when you photograph bright small objects such as fairly lights on a dark background.

The smaller the pinhole, the smaller the cone and the sharper the image. However, a smaller pinhole requires a longer exposure time because less light gets in the camera. It won't be a problem for a CG camera, but for practical real-world pinhole cameras, a longer exposure time increases the risk of producing a blurred image if the camera is not perfectly still or if the subject moves and is not ideal. The shorter the exposure time the better. There is a limit though to the size of the pinhole. When it gets very small (the size of the hole is about the same as the light's wavelength), light rays are diffracted. For a shoe-box sized pinhole camera, a pinhole of about 2 mm in diameter should produce optimum result (a good compromise between image sharpness and exposure time). Note that when the aperture is too large (figure 5 bottom), a single point on the image if you keep using the concept of point or discrete lines to represent light rays (for example point A or B in figure 5) appears multiple time on the image. A more accurate way of visualising what's happening in that particular case, is to imaging the footprints of the cones overlapping each over on the film (figure 6 bottom). As the size of the pinhole increases, the cones become larger and the amount of overlap increases. The fact that a point appears multiple time in the image (in the form of the cone's footprint or spot becoming larger on the film, which you can see as the color of the object at the light ray's origin being spread out on the surface of the film over a larger region rather than appearing as a singular point as it theoretically should) is what causes an image to be blurred. In photography, this effect is much more visible when you take a picture of very small and bright objects on a dark background such as fairy lights at night for instance (figure 8). The fairy lights are very small in frame, but they are so bright compared to other objects from the scene, that the spot or disk resulting from the intersection of the cone with the film plane, is clearly visible on the picture (all the other objects produce a similar effect but in contrast to the fairy lights it's simply not as visible). In photography, these disks (which are not always perfectly round in shape but explaining why is outside the scope of this lesson) are called circles of confusion (also known as disks of confusion, blur circles, blur spots, etc.).

To better understand the image formation process we have a created two short animations showing light rays from two disks passing through the camera's pinhole. In the first animation (figure 9), the pinhole is small and the image of the disk is sharp.

Figure 9: animation showing light rays passing through the pinhole and forming an image on the film plane. The image of the scene is inverted.

The second animation (figure 10) shows what happens when the pinhole is too large. For this animation we select a position on the disk we want to draw on the film plane, but rather than drawing the cone corresponding to all the rays passing through the pinhole for that point, we just draw a few rays contained within that cone of directions. Repeated multiple times for multiples points on the disks, we can see how the resulting points corresponding to the intersection of the rays with the film, form a blurred image of the disk. If we were to run this simulation for a very long time we would get a smooth result representing a perfectly blurred image of the two disks.

Figure 10: when the aperture or pinhole is too larger, a point from the geometry appears in multiple places on the film plane and the resulting image is blurred.

In conclusion, to produce a sharp image we need to make the aperture of the pinhole camera as small as possible to ensure than only a narrow beam of photons coming from one single direction enters the camera and hit the film or the sensor in one single point. However for our virtual camera model, you will see later in this chapter that the size of the pinhole is actually not going to be an issue.

Because the pinhole is very small, very little light can pass through the aperture and long exposure times are required to form an image. It is a limitation if you wish to take picture of moving objects for which very short exposure times are required if you don't want them to be blurred or in low light conditions. Of course the bigger the aperture the more light gets in, however as showed before this led to blurred images. The solution is to place a lens in front of the aperture to focus the rays back into one point on the film plane as shown in the adjacent figure. This lesson is only an introduction on pinhole cameras and not a thorough explanation on how cameras work and why are cameras equipped with lenses. More information on this topic can be found in the lesson on depth of field in the basic section (not yet released). However, as a note, and if you try to make the relation between how a pinhole camera and a modern camera work, it is important to know that lenses are used to make the aperture as large as possible allowing more light to get in the camera and therefore reducing exposure times. The role of the lens is to cancel the blurry look of the image we would get if we were using a pinhole camera with a large aperture by refocusing the light rays to a single point on the image plane. By combining the two, a large aperture and a lens, we get the best of both systems, smaller exposure times and sharp images (however the use of lenses introduces depth of field but as we mentioned before, this won't be studied or explained in the lesson). The great thing about pinhole cameras though, is that they don't require lenses and are therefore very simple to build and are also very simple to simulate in computer graphics.

In photography depth of field defines the distance between the nearest and the farthest object from the scene that appears sharp on the image. Pinhole cameras have an infinite depth of field. In other words, the sharpness of an object on the image does not depend on its distance to the camera (assuming the pinhole itself has the right diameter for the size of the camera). This is generally not the case of photographs produced with lens cameras. Computer graphics images are most of the time produced using a pinhole camera model, and similarly to real-world pinhole cameras, they have an infinite depth of field; all objects from the scene visible though the camera are rendered perfectly sharp. CGI has sometimes been criticized for producing images which are too clean and the use of this camera model has certainly a lot to do with it. Being just the result of how light rays interact with the lenses of the camera's (when lenses are used), depth of field can however be easily modelled using simple geometry and laws of optics. A whole lesson is dedicated to this topic in the basic section.

A CG Pinhole Camera Model

Now that we have understood the principle of the real pinhole camera lets see how we will apply this knowledge for designing a virtual camera model. When light rays emitted by a source intersect objects from the scene, they are reflected back in random directions. For each point of the scene visible by the camera, if we want to get a perfectly sharp image, ideally only one of these reflected rays will pass through the aperture of the pinhole camera and enter the box where the film will be then exposed. What we know then, is that each pixel of the frame (if we deal with a digital camera) or each very small area of the film paper corresponds to one light ray coming from one single direction. If we trace this ray (R) from the point where it struck the film into the scene (which is quite simple, we just trace a line from that point (A) through the aperture of the camera), R will intersect the object from the scene in one point (P) which is the point where the light ray was emitted from (light rays reflected back into space can be seen as a form of light emission). Our virtual camera model will follow the same principle. Rather than following the natural path of the light ray, form the object to the camera, we follow that path but in the other direction, from the camera to the object. For each pixel of the image we will trace a ray from that point through a point in space which represents the aperture of the camera (the center of projection). We will then test if this ray intersects any object of the scene and if it does, we will compute the color of the object at the point of intersection (P). Finally, we will assign this color to the pixel from which the ray was traced (remember from lesson 1 that if the ray doesn't intersect any geometry from the scene, then we simply return black or the background color). This process is illustrated in figure 11. Because rays in ray tracing follow the path of light rays (in reverse though) we speak of light transport, a term that you will often read when we will get to the lessons on shading.

Figure 11: in the real world, light travels from the light source to the pixel of the camera which we have represented in the first part of the animation. Green light is bounced back from the sphere and hit the pixel of the CCD at point A along ray R from point P. In CG we can simply follow the path of the ray the other way around. We start from A, shoot a ray to the scene and hit point P. We compute the color at the hit point and assign the pixel at A with this color.

Lets finish this paragraph with a few important notes: changing the distance between the film and the aperture changes the size of the image (which relates to the concept of focal in photography) and changing the size of the film paper allows the images to capture more or less of the scene (this concept of film size is important in computer graphics and often misunderstood). Finally, photographs produced by real world pinhole cameras are upside down which is confusing. This is happening because the film plane is located behind the center of projection, however this can be avoided if the projection plane lies on the same side as the scene as showed in figure 12. In the real world this is not possible, but in the virtual world of computers, constructing our camera that way is not a problem at all. The process of constructing primary or camera rays is simple: for a given pixel from the image, we trace a line from the center of projection (the camera's origin) to the center of that pixel. This line defines the ray direction and the camera's origin defines the ray's origin (the concept of ray will be presented in the next chapter). This process is repeated for every pixel of the frame. You also probably understand now why the size of the aperture is actually not a problem. In our virtual camera model rays are all emitted from the camera origin thus the aperture is reduced to a singular point (the center of projection) and the concept of aperture size in this model doesn't exist. Our CG camera model behaves as a perfect pinhole camera in a way, since to each pixel in the frame corresponds only one single light ray.

Figure 12: for our virtual camera, we can move the image plane in front of the aperture. That way, the projected image of the scene on the image plane is not inverted.

What's Next

In this chapter we have reviews some basic concepts about how cameras from the real world work (how an image is created on the film) and explained the principle of the pinhole camera. In the next chapter we will explain the concept of rays which we will be using to compute the color of each pixel from the digital image.

Chapter 1 of 5