Type II Error

An example of a false negative is when someone tests for coronavirus and receives a result that says they are not infected, but they actually are.

Consider a variable \(X\) with a Poisson distribution. A sample is taken, and a statistician wants to carry out the following hypothesis test, \(H_0: \lambda=9\) v.s. \(H_1:\lambda\neq9\), at the 5% significance level.

a) Find the critical region for this test.

b) Suppose that the true value of \(\lambda\) was later found to be 8, calculate the probability of a Type II error.

Solution

a) Since \(H_0: \lambda=9\) v.s. \(H_1:\lambda\neq9\), we are dealing with a two-tailed test.

Assume \(H_0\) is true, that is assume \(X\sim \text{Poi}(9)\).

Let \(X=c_1\) be the upper bound of the lower critical region. We want to find \(c_1\) such that \(\mathbb{P}(X \leq c_1)<0.0025\)

From the statistical tables,

\[\begin{align} \mathbb{P}(X \leq 4)&=0.0550>0.0025 \\ \mathbb{P}(X \leq 3)&=0.0212<0.0025 \end{align}\]

Therefore, \(c_1=3\).

Let \(X=c_2\) be the lower bound of the upper critical region. We want to find \(c_2\) such that \(\mathbb{P}(X \geq c_2)<0.0025\).

From the statistical tables,

\[\small{\begin{align} \mathbb{P}(X \geq 15)&=1-\mathbb{P}(X \leq 14)=1-0.9585=0.0415>0.0025 \\ \mathbb{P}(X \geq 16)&=1-\mathbb{P}(X \leq 15)=1-0.9780=0.0220<0.0025 \end{align}}\]

Therefore, \(c_2=16\).

So the critical region for this test is \({X\leq3}\) and \({X \geq 16}\).

b) Since we have the true value of \(\lambda=8\), we know that the null hypothesis is false so we can work out the probability of a type II error.

\(\begin{align} \mathbb{P}(\text{Type II error})&=\mathbb{P}(\text{accepting } H_0 \text{ when } H_0 \text{ is false}) \\ &=\mathbb{P}(4\leq X\geq 15 \mid H_0 \text{ is false}) \end{align}\)

Given that the true value of \(\lambda=8\),

\(\begin{align} \mathbb{P}( \text{ Type II error })&=\mathbb{P}(4 \leq X \geq 15\mid \lambda=8) \\ &=\mathbb{P}(X \geq 15 \mid \lambda=8)-\mathbb{P}(X \leq 3 \mid \lambda=8) \\ &=0.9918-0.0424=0.9494. \end{align}\)

Suppose someone claims that the average height of males in the U.S. is normally distributed with a mean of 70 inches and a standard deviation of 3 inches.

A statistician decides then to take a random sample of 36 males from the population of the U.S. in order to test this claim.

Let the random variable \(X\) denote the height of a male.

a) Using a significance level of 5%, find the critical region for this test.

b) Given that the average height was in fact 65 inches, find the probability that the person's claim is wrongly accepted.

Solution

a) We define the null hypothesis

\[H_0: \mu=70 \quad \text{v.s.} H_1: \mu\neq70.\]

Assume \(H_0\), then since \(X\) denotes the height of a male, the average height of males in the U.S. is distributed so that \(\bar{X} \sim N(70, 3^2/36)\).

Since we want to test using the mean of a normal random variable, to simplify things, we can use the result,

If \(\bar{X} \sim N(\mu,\sigma^2)\), then \(Z=\frac{\bar{X}-\mu}{\frac{\sigma}{\sqrt n}} \sim N(0,1)\).

Standardize this \(\bar{X}\) variable: \( Z=\frac{\bar{X}-70}{\frac{3}{\sqrt(36)}} = \frac{\bar{X}-70}{\frac{1}{2}}=2(\bar{X}-70)\) where the random variable \(Z \sim N(0, 1)\).

With a 5% significance level, since we have a two-tailed hypothesis test, we need 2.5% on each tail.

From the statistical tables, the critical region for \(Z\) is

\(Z > 1.9600\) or \(Z<-1.9600\)

So the critical values for \(\bar{X}\) are given by

\[2(\bar{X}-70)=\pm 1.96\]

\[\therefore \bar{X} = 69.02 \quad \text{and} \quad \bar{X} = 70.98\]

So the critical region for \(\bar{X}\) is \(\bar{X} <69.02\) or \(\bar{X} >70.98\)

b) If the person's claim for the average male height is accepted although the actual mean height turns out to be different, that is a Type II error.

\[\begin{align} \mathbb{P}(\text{Type II error})&=\mathbb{P}(69.02 \leq \bar{X} \leq 70.98 \mid \mu=65) \\ &=\mathbb{P}(\bar{X} \leq 70.98 \mid \mu=65)-\mathbb{P}(\bar{X} \leq 69.02 \mid \mu=65) \\ &=0.9769-0.9099\\&=0.067. \end{align}\]

Suppose a random variable \(X\) has a geometric distribution. A statistician wants to test the hypothesis \[H_0: p=0.05\quad \text{v.s.} \quad H_1: p\neq0.05\] using a 1% significance level.

a) Find the critical region for this test.

b) Now, given that \(p=0.03\), find the power of this test.

Solution

a) Assume we are under the null hypothesis, \(H_0\), so that \(X \sim \text{Geo}(0.05)\). Since this is a two-tailed test at the 1% significance level, if \(X=c_1\) is the lower bound of the upper critical region then, we need to find \(c_1\) such that \[\mathbb{P}(X \geq c_1)<0.005.\]

From the distribution of a geometric random variable, we have

\[\begin{align} (1-0.05)^{c_1-1}&<0.005 \\ c_1-1&>\frac{\ln(0.005)}{\ln(0.95)} \\ c_1&>104.29454 \end{align}\]

So \(c_1=104\) which gives an upper critical region of \(X \geq 104.\)

If \(X=c_2\) is the upper bound of the lower critical region then, we need to find \(c_2\) such that

\[\mathbb{P}(X \leq c_2)<0.005.\]

\[\begin{align} 1-(1-0.05)^{c_2}&<0.005 \\ 0.95^c_2&>0.995 \\ c_2&<\frac{\ln(0.995)}{\ln(0.95)} \\ c_2&<0.0977 \end{align}\]

So \(c_2=0.1\) which gives a lower critical region of \(X \leq 0.01.\)

b) The power of the test can be calculated via, \[ \begin{align} \text{Power }&= \mathbb{P}(H_0 \text{ is rejected} \mid p=0.03) \\ &=\mathbb{P}(X \leq 0.1 \mid p=0.03)+\mathbb{P}(X \geq 104 \mid p=0.03) \\ &=1-(1-0.03)^{0.1}+(1-0.03)^{104}=0.04513 \end{align}\]