Probability Density Function

Probability density function graph

First of all, what is a probability density function?

That looks more complicated than it actually is. Let's relate it to the graph of a function.

Take the function

\[ f_X(x) = \begin{cases} 0.1 & 1 \le x \le 11 \\ 0 & \text{otherwise} \end{cases} \]

as seen in the graph below.

Fig. 1 - Graph of a potential probability density function.

Let's check it for the properties of a probability density function. It is certainly at least always zero. The area under the curve is \(1\) since that area is just a rectangle with height \(0.1\) and width \(10\). And lastly, you can represent the probability as an area. For example, if you wanted to find \(P(5<X<7)\) you could do so by finding the area of the rectangle in the graph below, getting that \(P(5<X<7) = 0.2\).

Fig. 2 - \(P(5<X<7) = 0.2\)

So \(f_X(x)\) is a probability density function. If you were to graph the probability curve, you would need to integrate it, giving you

\[ P(a<X<b) = \begin{cases} 0 & a \text{ and } b \le 1 \\ 0.2(b-1) & a<1<b \le 11 \\ 0.2(b-a) & 1 \le a \le b \le 11 \\ 0.2(11-a) & 1 <a <11 < b \\ 1 & 11 \le a < b \end{cases} \]

That certainly seems like a lot of cases, but you can see it much more easily by looking at the graph below.

Fig. 3 - Graph of probabilities related to \(f_X(x)\).

Notice that the minimum height of the graph above is \(0\), and the maximum height of the graph is \(1\). This makes sense because probabilities are always at least zero and at most one.

Probability density function properties

Using the definition of the probability density function, you can see an important property of them:

\[P(X=a) = 0.\]

It also doesn't matter if you use strict inequalities with continuous density functions:

\[ P(X<a) = P(X\le a).\]

Both of those properties come from the fact that

\[ P(a<X<b) = \int_a^b f_X(x) \, \mathrm{d} x .\]

You might ask if the probability density function can be greater than \(1\). Sure it can! The integral of the function still needs to be equal to \(1\), but the probability density function can take on values larger than that as long as it is also at least zero. One example of this is the probability density function

\[ f_X(x) = \begin{cases} 2 & 0 \le x \le \dfrac{1}{2} \\ 0 & \text{otherwise} \end{cases} .\]

This function is always at least zero, it is integrable, and the integral is \(1\), so it could be a probability density function for a continuous random variable \(X\). Don't confuse the probability density function for actual probabilities!

Probability density function of normal distribution

One of the probability density functions you will see often is the normal distribution. You can see the graph of the standard normal distribution probability density function below.

Fig. 4 - The standard normal distribution.

Just like with other probability density functions, the area under the curve of the standard normal distribution is \(1\).

Probability density function example

Let's look at some examples.