The world’s leading publication for data science, AI, and ML professionals.

Clarifying Type I and Type II Errors in Hypothesis Testing

Deciphering the differences between alpha and beta

Image Source
Image Source

When you’re conducting statistical tests to decide whether you think the argument is true or false, you’re hypothesis testing. The null hypothesis is the initial statement that you are testing. The null hypothesis is believed to be true unless there is overwhelming evidence to the contrary. One common example of this is when you assume that two groups are statistically different from each other.

However, there are times when scientists reject the null hypothesis when they should not have rejected it. The reverse could also happen if the null hypothesis is not rejected when it should have been. Data scientists refer to these errors as Type I and Type II errors, respectively. In this blog post, I’m going to delve deeper into Type I and Type II errors.

Type I Errors – False Positives (Alpha)

There will almost always be a possibility of wrongly rejecting a null hypothesis when it should not have been rejected while performing hypothesis tests. Data scientists have the option of selecting an Alpha (𝛼) confidence level threshold that they will use to accept or reject the null hypothesis. This confidence threshold, which is in other words a level of trust, is also the likelihood that you will reject the null hypothesis when it is actually valid. This case is a type I error, which is more generally referred to as a false positive.

In hypothesis testing, you need to decide what degree of confidence, or trust, for which you can dismiss the null hypothesis. If a scientist were to set alpha (𝛼) =.05, this means that there is a 5 percent probability that they would reject the null hypothesis when it is actually valid. Another way to think about this is that you would expect the hypothesis to be rejected once, simply by chance, if you repeated this experiment 20 times. Generally speaking, an alpha level of 0.05 is adequate to show that certain findings are statistically significant.

Type II Errors – False Negatives (Beta)

Beta (β) is another type of error, which is the possibility that you have not rejected the null hypothesis when it is actually incorrect. Type II errors are also known as false negatives. Beta is linked to something called Power, which, given that the null hypothesis is actually false, is the likelihood of rejecting it. When planning an experiment, researchers will always select the power level they want and get their Type II error rate from that.

Is one more important than the other?

Various situations allow researchers to mitigate one form of Error over the other. The two types of error are inversely related to each other; decreasing type I errors will increase type II errors, and vice versa. To decide when a type I or type II error would be safer, let’s go through a couple of scenarios.

Imagine that you are on a jury and that you need to determine if an individual is going to be sent to jail for a crime. Since you don’t know the truth as to whether or not this person committed a crime, which would be worse, a type I or type II error? I hope you say that a type I error is going to be worse. A type I error would suggest that, if they were really not guilty, you would send them to jail! The jury has dismissed the null hypothesis that the defendant is innocent while he has not committed any crime. You would also not want to make a type II error here because this would mean that someone has actually committed a crime and the jury is letting them get away with it.

Let’s take another example of a medical situation. A patient with multiple migraine headaches is referred to the doctor for an MRI head scan. The doctor believes that a brain tumor may be present in the patient. Is it going to be worse for this situation to have a type I or type II error? Let’s hope you said that a Type II error would be worse. A type II error would mean that there is a brain tumor in the patient, but the doctor insists that there is nothing wrong with them! In other words, the null hypothesis is that the person does not have a brain tumor, and this hypothesis is not denied. This implies that, even though they are genuinely far from it, the person is diagnosed as healthy.

As researchers design experiments and make choices about the degrees of alpha level and power, they need to weigh the risks of Type I and Type II errors in order to prepare for whatever type of error they want to mitigate.

I hope this post helped clarify the differences between type I and type II errors in Hypothesis Testing. Thank you for reading!

LinkedIn


Related Articles