To listen to the lecture, click here.
Two more topics: two-sided tests and power
A typical hypothesis test has Ha as m > m0 or m < m0. But you could have the alternative simply be that m does not equal m0. That is called a two-sided test or a two-sided alternative.
With a two-sided test, instead of comparing the p-value to a, you compare the p-value to a/2.
Remember that the significance level is the probability of making a type I error. Now, you can make a type I error on either side. So instead of giving yourself a 5 percent probability on one side, you give yourself a 2.5 percent probability on each side. Thus, for a given significance level, it is harder to reject the null hypothesis against a two-sided alternative.
Power relates to type II error. If B is the probability of type II error, then Power is 1-B. Higher power is better.
There are two ways to increase power. One is to set a higher significance level (not a good idea--increases the risk of type I error). The other is to increase the sample size (reduces the probability of both types of errors).
Power can only be computed against a specific alternative, not against an inequality.
The steps in computing power
Pretend that the null hypothesis is true, and find the cutoff point, in natural units, where you would just fail to reject the null hypothesis. Call this X*.
Example: Suppose that we want to test the hypothesis that the mean weight of NFL linebackers today is the same as it was ten years ago against the specific alternative hypothesis that the mean weight is 5 pounds higher. Take it as given that our significance level is 5 percent. Suppose that the mean weight ten years ago was 238 pounds. Suppose that we know that the standard deviation is 8 pounds. Finally, suppose that we take a sample of 16 linebackers today.
First, pretend that the null hypothesis is true and find X*. The null hypothesis that m = 238, and the alternative is that m is greater than 238. To find X*, we first take invnorm (.05) = 1.64, which is the number of standard deviations away from the mean we have to be to reject the null hypothesis. To convert this to natural units, we use the Z transformation:
1.64 = (X* - 238)/[8/sqrt(16)]; X* = 241.28
Second, we pretend that the alternative hypothesis is true, and the mean weight is 243 pounds. What would be the p-value for 241.28? The Z-transformation gives
Z = (241.28 - 243)/[8/sqrt(16)] = -0.96
Taking invnorm (-.96) we get .17, so that the power is .83.