Degrees of Freedom

Degrees of freedom meaning

Many tests use degrees of freedom, but here you will see degrees of freedom as it relates to Chi Squared Tests. In general, the degrees of freedom is a way to measure how many test statistics you have calculated from the data. The more test statistics you have calculated using your sample, the less freedom you have to make choices with your data. Of course, there is a more formal way to describe these constraints as well.

Let's look at an example to see what that means in practice.

The number of constraints will also depend on the number of parameters you need to describe a distribution, and whether or not you know what these parameters are.

Next, let's look at how the constraints relate to degrees of freedom.

Degrees of freedom formula

For most cases, the formula

degrees of freedom = number of observed frequencies - number of constraints

can be used. If you go back to the example with the four sided die above, there was one constraint. The number of observed frequencies is \(4\) (the number of sides on the die. So the degrees of freedom would be \(4-1 = 3\).

There is a more general formula for the degrees of freedom:

degrees of freedom = number of cells (after combining) - number of constraints.

You are probably wondering what a cell is and why you might combine it. Let's look at an example.

You will usually only combine adjoining cells in your tables of data. Next, let's look at the official definition of degrees of freedom with the Chi-Squared distribution.

Degrees of freedom definition

If you have a random variable \(X\) and want to do an approximation for the statistic \(X^2\), you would use the \(\chi^2\) family of distributions. This is written as

\[\begin{align} X^2 &= \sum \frac{(O_t - E_t)^2}{E_t} \\ &= \sum \frac{O_t ^2}{E_t} -N \\ & \sim \chi^2, \end{align}\]

where \(O_t\) is the observed frequency, \(E_t\) is the expected frequency, and \(N\) is the total number of observations. Remember that the Chi-Squared tests are only a good approximation if none of the expected frequencies is below \(5\).

The \(\chi^2\) distributions are actually a family of distributions that depend on the degrees of freedom. The degrees of freedom for this kind of distribution are written using the variable \(\nu\). Since you may need to combine cells when using \(\chi^2\) distributions, you would use the definition below.

There will be cases where cells won't be combined, and in that case, you can simplify things a bit. If you go back to the four sided die example, there are \(4\) possibilities that could come up on the die, and these are the expected values. So for this example \(\nu = 4 - 1 = 3\) even if you are using a Chi-Squared distribution to model it.

To be sure you know how many degrees of freedom you have when using the Chi-Squared distribution, it is written as a subscript: \(\chi^2_\nu \).

Degrees of freedom table

Once you know that you are using a Chi-Squared distribution with \(\nu\) degrees of freedom, you will need to use a degrees of freedom table so that you can do hypothesis tests. Here is a section out of a Chi-Squared table.

Table 3. Chi-Squared table.

The first column of the table contains the degrees of freedom, and the first row of the table are areas to the right of the critical value.

The notation for a critical value of \(\chi^2_\nu\) which is exceeded with probability \(a\%\) is \(\chi^2_\nu(a\%)\) or \(\chi^2_\nu(a/100)\).

Let's take an example using the Chi-Squared table.

There is a second use for the table, as demonstrated in the next example.

Of course, a table can't list all of the possible values. If you need a value which is not in the table, there are many different statistics packages or calculators that can give you Chi-Squared table values.

Degrees of freedom t-test

The degrees of freedom in a \(t\)-test is calculated depending on if you are using paired samples or not. For more information on these topics, see the articles T-distribution and Paired t-test.