Hypothesis testing is a common statistical method that allows us to make inferences about a population parameter based on a sample statistic. For example, we may want to test whether the mean height of a group of students is different from the national average, or whether the proportion of smokers in a city has increased over time.
To perform a hypothesis test, we need to specify two competing hypotheses: the null hypothesis and the alternative hypothesis. The null hypothesis is the default assumption that there is no effect or difference in the population, while the alternative hypothesis is the opposite claim that there is some effect or difference.
For example, if we want to test whether the mean height of a group of students is different from the national average of 170 cm, we can set up the following hypotheses:
- Null hypothesis (H0): The mean height of the students is equal to 170 cm.
- Alternative hypothesis (HA): The mean height of the students is not equal to 170 cm.
To test these hypotheses, we need to collect a random sample of students and measure their heights. Then, we need to calculate a test statistic that measures how far the sample mean is from the hypothesized population mean. For example, we can use a t-test statistic that follows a t-distribution with a certain degree of freedom.
The test statistic tells us how likely it is to observe such a sample mean if the null hypothesis were true. However, we cannot directly compare the test statistic to the null hypothesis, because they are in different units. Therefore, we need to convert the test statistic into a p value, which is a probability value that ranges from 0 to 1.
What is the P Value?
The p value is the probability of obtaining a test statistic at least as extreme as the one observed, assuming that the null hypothesis is true. The smaller the p value, the less likely it is to observe such an extreme test statistic by chance, and the more evidence we have against the null hypothesis.
For example, suppose we collect a sample of 25 students and find that their mean height is 173 cm, with a standard deviation of 5 cm. Using a t-test, we can calculate that the test statistic is 2.37. The p value for this test statistic is 0.026, which means that there is only a 2.6% chance of observing such a high test statistic if the null hypothesis were true.
What is the Rejection Region?
The rejection region is a range of values of the test statistic that leads us to reject the null hypothesis in favor of the alternative hypothesis. The rejection region depends on two factors: the significance level and the type of test.
The significance level, denoted by α, is the maximum probability of making a type I error, which is rejecting the null hypothesis when it is actually true. The significance level is usually set at 0.05 or 0.01, depending on how confident we want to be in our decision.
The type of test determines whether we use a one-tailed or a two-tailed rejection region. A one-tailed test is used when we have a directional alternative hypothesis, such as “the mean height of the students is greater than 170 cm”. A two-tailed test is used when we have a non-directional alternative hypothesis, such as “the mean height of the students is not equal to 170 cm”.
For example, if we use a one-tailed test with α = 0.05 and HA: The mean height of the students is greater than 170 cm, then our rejection region will be all values of the test statistic that are greater than or equal to 1.71 (the critical value for a t-distribution with 24 degrees of freedom and α = 0.05). If we use a two-tailed test with α = 0.05 and HA: The mean height of the students is not equal to 170 cm, then our rejection region will be all values of the test statistic that are less than or equal to -2.06 or greater than or equal to 2.06 (the critical values for a t-distribution with 24 degrees of freedom and α = 0.025 for each tail).
The p value and the rejection region are two ways of making a statistical decision based on our test statistic. They are related by the following rule:
- If the p value is less than or equal to α, then we reject H0 and accept HA.
- If the p value is greater than α, then we fail to reject H0 and do not accept HA.
Alternatively,
- If the test statistic falls in the rejection region, then we reject H0 and accept HA.
- If the test statistic falls outside the rejection region, then we fail to reject H0 and do not accept HA.
For example, using the same sample data as before, we can use either the p value or the rejection region to make our decision. If we use a one-tailed test with α = 0.05 and HA: The mean height of the students is greater than 170 cm, then we can compare our p value of 0.026 to α, or our test statistic of 2.37 to the critical value of 1.71. In both cases, we will reject H0 and accept HA, because the p value is less than α and the test statistic falls in the rejection region. If we use a two-tailed test with α = 0.05 and HA: The mean height of the students is not equal to 170 cm, then we can compare our p value of 0.026 to α/2, or our test statistic of 2.37 to the critical values of -2.06 and 2.06. In both cases, we will also reject H0 and accept HA, because the p value is less than α/2 and the test statistic falls in the rejection region.
Conclusion
The rejection region and the p value are two methods of making a statistical decision based on a hypothesis test. They are related by a simple rule that compares the p value to the significance level or the test statistic to the critical value. Both methods will lead to the same conclusion, but using the p value has some advantages over using the rejection region, such as:
- It does not require us to look up the critical value for different significance levels and types of tests.
- It gives us an idea of how strong the evidence is against the null hypothesis, not just whether to reject it or not.
Therefore, using the p value is more common and preferred in most statistical applications. However, understanding the rejection region can help us better understand the logic and intuition behind hypothesis testing.
