Mathematical Foundations of Monte Carlo Methods (Probability Distribution: Part 2)

Mathematical Foundations of Monte Carlo Methods

Contents

A Quick Introduction to Monte Carlo Methods

Random Variables and Probability

Probability Distribution: Part 1

Probability Properties

Introduction to Statistics

Expected Value

Variance and Standard Deviation

Probability Distribution: Part 2

Sampling Distribution

Probability Density Function (PDF) and Cumulative Distribution Function (CDF)

Expected Value of the Function of a Random Variable: Law of the Unconscious Statistician

The Inverse Transform Sampling Method

Estimators

In chapter 2, we have mentioned the normal (or gaussian) probability distribution. We now have the knowledge needed to introduce and understand its equation. Why studying the normal distribution? This distribution is very common in nature. For example the adult height in any adult given population generally follows (more or less) a normal distribution. We will see it reapparing in the next chapter as well.

If you reproduce these curves for yourself, you will probably get a better sense of what the parameters of the equation do. You can use Grapher on Mac and Gnuplot on Linux.

Gaussian or normal distribution have a typical bell-shaped curve (see figure 1). The equation for this distribution is a bit complex:

$$p(x) = \mathcal{N}(\mu, \sigma) = {\dfrac{1}{\sigma \sqrt {2 \pi} } } e^{-{\dfrac{(x -\mu)^2}{2\sigma^2}}}.$$

Figure 1:a normal distribution with standard deviation ($\sigma = 1$) and expectation ($\mu = 0$).

You should already be familiar with the $\sigma$ term (the greek letter sigma) which is the distribution's standard deviation (see the previous chapter if you need a refresher on what the standard deviation is). The term $\mu$ (the greek letter mu) is the distribution's expectation (a concept we have also introduced in the previous chapter). Note that the curve is symmetrical around $\mu$. By changing the value of these parameters we can obtain all sort of bell-shaped distributions as pictured in figure 2. The normal distribution function is generally denoted $\mathcal{N}(\mu, \sigma)$ since mean and variance are the two parameters of the model. $\mathcal{N}(0,1)$ is called the standard normal distribution (figure 1).

Advanced: add in a future revision of the lesson show that the expected value of the standard normal distribution is 0, area under the curve is 1 and standard deviation is 1. Note in the figure below that when the standard deviation is smaller the curve is higher. The area under the curve is 1 not matter which standard deviation you chose.

Figure 2: example of gaussian distribution for different values of the standart deviation ($\sigma$) and different values of the mean ($\mu$).

If you read more about statistics you may come across the terms skew and kurtosis. In the image above we have drawn a normal distribution in red (with a standard deviation of 2 and a mean of 5). When the curve is perfectly symmetrical around the mean (the vertical cyan line) then we say that the skew of the distribution is 0. When the distribution has a larger tail to the left, the skew is negative (and the skew is positive when the tail is larger to the right). The term kurtosis designates how pointy or smooth the curve is compared to a normal distribution (right). A curve which is narrower than the a normal distribution is said to have a positive kurtosis. These are not so important (especially in computer graphics) but these two parameters will help us in the next chapter to compare how close our distribution of data is to a "perfect" normal distribution.

In the next chapter, we will learn about the concepts of PDF (probability density function) which is derived from the concept of probability distribution, and CDF (cumulative distribution function). These two concepts are very important in computer graphics.