Hypothesis Test for Regression Slope

Meaning of the Hypothesis Test for Regression Slope

Suppose that to find the relationship between two variables, you have used linear regression to obtain an equation \[\hat{y}=\alpha+\beta x.\]

In theory, this equation should allow you to predict values of \(y\), by evaluating at \(x\), that is, \(y\approx\hat{y}(x)\).

But how can you be confident that the linear regression equation obtained is good at predicting \(y\) values? As mentioned at the beginning, a hypothesis test can help you.

Hypothesis testing is based on calculating how likely it is to obtain a sample like yours, if certain conditions are assumed, in this case, assuming the regression slope obtained, what is the probability of obtaining the sample in question.

Importance of Hypothesis Test for Regression Slope

Whenever you use linear regression to model the behavior of two datasets that are related, the regression slope that you get is an estimate of how one data changes regarding the other.

Normally, this linear regression equation changes each time you take a different sample, so it makes sense to ask yourself if the actual slope value of the population is similar to the one you get from the sample using linear regression.

The following images show the scatter plots of \(2\) sets of data with their respective regression line.

A good regression line should allow you to predict \(y-\)values knowing the \(x-\)values quite accurately. Looking at the first image, you can notice that since the points are close to the line, then the regression line is good.

Fig. 1 - A scatter plot with a good regression line

On the other hand, in the second image, several values are far from the values predicted by the regression line. For this reason, you can say that the regression line is not so good.

Fig. 2 - A scatter plot with a bad regression line

In situations like the graph above, it makes sense to doubt how good the obtained regression line is.

Hypothesis Test for Regression Coefficients

There are many hypothesis tests that can be performed on the slope of the regression line. These consist of having a null hypothesis, which it can be

\[H_0:\; \beta=\beta_0,\]

that is, that the regression slope is equal to a certain value.

While the alternative hypothesis will be some form of negation of the null hypothesis, such as

• \( H_a:\;\beta>\beta_0 \);

• \(H_a:\; \beta<\beta_0 \); or

• \( H_a:\; \beta\neq\beta_0 \).

Although the slope of a regression line can have many values, hypothesis testing generally only focuses on answering: Is the slope different from zero? If it is different from zero, then you will be able to use it to make predictions. Therefore, this article will only focus on making this type of hypothesis.

Why can't you use a regression line with zero slope to make predictions? A regression line with zero slope means that the data for \(y\) does not depend on \(x\), in other words, knowing the value of \(x\) does not allow you to predict the value of \(y\) using the regression line. This means the regression line is not useful.

Conditions for Hypothesis Test for Regression Slope

To be able to make inference about the coefficients of the regression line, you must make sure that your data meets the following conditions:

1. Linearity: The scatter plot of the data looks straight.

2. Independence: The residuals must be independent (see the article Residuals for more information about this).

3. Equal variance: The standard deviation of the \(y\)-values should be nearly equal for all values of \(x\).

4. Normal population: The \(y\)-values are distributed normally for any value of \(x\).

Methods of Hypothesis Test for Regression Slope

Recall that in this article you will only learn how to perform the hypothesis test to prove that the slope of the regression line is non-zero. So, the procedure is as follows:

Step 1. State the hypotheses.

The null hypothesis and the alternative hypothesis are given by

\[\begin{align} &H_0\; :\beta=0 \\ &H_a:\;\beta\neq 0. \end{align}\]

The null hypothesis states that the slope is zero, which is equivalent to saying that there is no useful linear relationship between \(x\) and \(y\) while the alternative hypothesis states that there is a useful linear relationship.

Step 2. Determine a significance level to use.

Normally, the significance level \(\alpha\) is taken as \(0.05\), but you can also consider \(0.01\), or \(0.1\).

Step 3. Find the test statistic and the corresponding \(p-\)value.

For this step, you need the standard error of the slope, the slope of the linear regression, the degrees of freedom (for samples having \(n\) pairs of data, the degrees of freedom are \(n-2\)) and the \(p-\)value associated to the test statistic.

The test statistic is given by

\[t=\frac{b}{s_b},\]

where \(b\) is the slope of the sample regression line, and the standard error \(s_b\) is given by

\[s_b=\frac{s_e}{\sqrt{\sum\limits_{i=1}^n(x_i-\mu_x)^2}}\]

where

\[s_e=\sqrt{\frac{\sum\limits_{i=1}^n(y_i-\hat{y})^2}{n-2}}.\]

You will also need the degrees of freedom for the \(t\)-distribution. Since it is paired data (the value of \(x\) is paired with a value of \(y\)), there are \(n-2\) degrees of freedom.

Step 4. Interpret results.

If the result obtained in the sample is unusual, given the null hypothesis, then the null hypothesis is rejected.

This step involves comparing the \(p\)-value obtained with the significance level, and the null hypothesis is rejected if the \(p\)-value is less than the significance level. Otherwise you will be unable to reject the null hypothesis.

Example of Hypothesis Test for Regression Slope