Residuals

Residuals in Math

For instance, assuming you want to find out how climate changes affect yield from a farm. You may specify climate variables in the model such as rainfall and temperature. However, other factors such as land size cultivated, and fertilizer use, among others, also affect farm yield. Hence, the question becomes, “is the model accurately predicting the level of yield considering climate changes as an explanatory variable?”. So how do you measure how much impact a given factor has? Let's look at a short and informal definition of a residual.

You can lean on the size of the residual to inform you about how good your prediction model is. That means you consider the value of the residual to explain why the prediction is not precisely as the actual.

In mathematics, residual value is usually used in terms of assets and in statistics (basically, in regression analysis as discussed in previous sections). The worth of an asset after a specified use-time explains the residual value of the asset.

So, formally you can define residuals as below.

Definition of Residual

The residual is the vertical distance between the observed point and the predicted point in a linear regression model. A residual is termed as the error term in a regression model, though it is not an error, but the difference in the value. Here is the more formal definition of a residual in terms of a regression line.

Mathematically, you can estimate the residual by deducting the estimated values of the dependent variable \((\hat{y})\) from the actual values given in a dataset \((y)\).

The residual is represented by \(\varepsilon \). That will mean

\[\varepsilon =y-\hat{y}.\]

The predicted value \((\hat{y})\) is obtained by substituting \(x\) values in the least-square regression line.

Residuals for data points

In the above graph, the vertical gap between a data point and the trendline is referred to as residual. The spot the data point is pinned determines whether the residual will be positive or negative. All points above the trendline show a positive residual and points below the trendline indicate a negative residual.

Residual in Linear Regression

For simplicity's sake let's look at residuals for bivariate data. In linear regression, you include the residual term to estimate the margin of error in predicting the regression line which passes through the two sets of data. In simple terms, residual explains or takes care of all other factors that may influence the dependent variable in a model other than what the model states.

Residuals are one way to check the regression coefficients or other values in linear regression. If the residual plot some unwanted patterns, then some values in the linear coefficients cannot be trusted.

You should make the following assumptions about the residuals for any regression model:

Assumptions of Residuals

• They have to be independent – no one residual at a point influences the next point’s residual value.

• Constant variance is assumed for all residuals.

• The mean value of all residuals for a model should equate to \(0\).

• Residuals should be normally distributed/follow a normal distribution – plotting them will give a straight line if they are normally distributed.

Residual Equation in Math

Given the linear regression model which includes the residual for estimation, you can write:

\[y=a+bx+\varepsilon ,\]

where \(y\) is the response variable (independent variable), \(a\) is the intercept, \(b\) is the slope of the line, \(x\) is

the explanatory variable (dependent variable) and \(\varepsilon\) is the residual.

Hence, the predicted value of \(y\) will be:

\[\hat{y} = a+bx .\]

Then using the definition, the residual equation for the linear regression model is

\[\varepsilon =y-\hat{y}\]

where \(\varepsilon\) represents residual, \(y\) is the actual value and \(\hat{y}\) is the predicted value of y.

This equation for residuals will be helpful in finding residuals from any given data. Note that, the order of subtraction is important when finding residuals. It is always the predicted value taken from the actual value. That is

residual = actual value – predicted value.

How to Find Residuals in Math

As you have seen, residuals are errors. Thus, you want to find out how accurate your prediction is from the actual figures considering the trendline. To find the residual of a data point:

• First, know the actual values of the variable under consideration. They may be presented in a table format.

• Secondly, identify the regression model to be estimated. Find the trendline.

• Next, using the trendline equation and the value of the explanatory variable, find the predicted value of the dependent variable.

• Finally, subtract the estimated value from the actual given.

This means if you have more than one data point; for example, \(10\) observations for two variables, you will be estimating the residual for all \(10\) observations. That is \(10\) residuals.

You can understand it more clearly by taking a look at an example.

You can see below how to plot the residuals in the graph.

Residual Plot

The residual plot measures the distance data points have from the trendline in the form of a scatter plot. This is obtained by plotting the computed residual values against the independent variables. The plot assists you to visualize how perfectly the trendline conforms to the given data set.

Fig. 1. Residuals without any pattern.

The desirable residual plot is the one which shows no pattern and the points are scattered at random. You can see from the above graph, that there is no specific pattern between points, and all the data points are scattered.

A small residual value results in a trendline that better fits the data points and vice versa. So larger values of the residuals suggest the line is not the best for the data points. When the residual is \(0\) for an observed value, it means that the data point is precisely on the line of best fit.

A residual plot can at times be good to identify potential problems in the regression model. It can much easier to show the relationship between two variables. The points far above or below the horizontal lines in residual plots show the error or unusual behavior in the data. And some of these points are called outliers regarding the linear regression lines.

Considering the same example used above, you can plot the residual values below.

From below, you can see how to work out the residual problem for different scenarios.

Residual Examples in Math

You can understand how to calculate residuals more clearly by following the residual examples here.

Consider another example to find the predicted values and residuals for the given data

The following example will show the plotting of residuals in the graph.