The world’s leading publication for data science, AI, and ML professionals.

Hypothesis Testing Explained as Simply as Possible

One of the most important concepts for Data Scientists

Image Credits: PIRO4D from Pixabay
Image Credits: PIRO4D from Pixabay

Table of Content

  1. Introduction
  2. Terminology
  3. Reject or Do not Reject?
  4. What is the point of Significance Testing?
  5. Steps for Hypothesis Testing

Introduction

If you’ve heard of the terms null hypothesis, p-value, **** and alpha but don’t really know what they mean or how they’re related then you’ve come to the right place! And if you’ve never heard of these terms, I urge you to read through this article as this is an essential topic to understand.

I’ll start with a simple example:

Imagine that you and your friend play a game. If a coin lands on heads, you win $5 and if it lands on tails he wins $5.

Image created by author
Image created by author

Let’s say the first two coin tosses landed on tails, meaning your friend won $10. Should you be worried that he’s using a rigged coin? Well, the probability of the coin landing on tails two times in a row is 25% (see above) which is not unlikely.

Image created by author
Image created by author

What if the coin landed on tails six times in a row? The probability of that occurring is approximately 1.56% (see above), which is highly unlikely. At this point, it would be fair to assume that the coin is rigged. Typically, one would set a threshold, usually 5%, to determine if an event occurred by chance or not (if you learned this before, this is known as the alpha!)

Terminology

To understand hypothesis testing, there’s some terminology that you have to understand:

  • Null Hypothesis: the hypothesis that sample observations result purely from chance. The null hypothesis tends to state that there’s no change.
  • Alternative Hypothesis: the hypothesis that sample observations are influenced by some non-random cause.
  • P-value: the probability of obtaining the observed results of a test, assuming that the null hypothesis is correct; a smaller p-value means that there is stronger evidence in favor of the alternative hypothesis.
  • Alpha: the significance level; the probability of rejecting the null hypothesis when it is true – also known as Type 1 error.

I’ll use the coin example again so that you can understand these terms better:

  • The null hypothesis in our example is that the coin is a fair coin and that the observations are purely from chance.
  • The alternative hypothesis would then be that the coin is not fair, and thus, the observations did not happen by chance.
  • The p-value in the scenario of flipping tails 2 times in a row is 25% and 6 times in a row is 1.56%.
  • The alpha or level of significance would be 5%.

Reject or Do not Reject?

The main rule in determining whether you reject the null is simple, PGADRN

If the P-value is Greater than the Alpha, Do not Reject the Null.

In the case of flipping tails 2 times in a row, we would not reject the null since 25% > 5%. However, in the case of flipping tails 6 times in a row, we would reject the null since 1.56% < 5%.

What is the point of Significance Testing?

So now that you understand the use of hypothesis testing through the coin toss example, know the relevant terminology, and know the main rule to determine whether to reject the null or not, let’s dive into significance testing.

What is the point of significance testing? It’s used to determine how likely or unlikely a hypothesis is for a given sample of Data. The last part of the statement, ‘for a given sample of data’ is key because more often than not, you won’t be able to get an infinite amount of data or data that represents the entire population.

Steps for Hypothesis Testing

Here are the steps to performing a hypothesis test:

  1. State your null and alternative hypotheses. To reiterate, the null hypothesis typically states that everything is as normally was – that nothing has changed.
  2. Set your significance level, the alpha. This is typically set at 5% but can be set at other levels depending on the situation and how severe it is to committing a type 1 and/or 2 error.
  3. Collect sample data and calculate sample Statistics.
  4. Calculate the p-value given sample statistics. Once you get the sample statistics, you can determine the p-value through different methods. The most common methods are the T-score and Z-score for normal distributions. Learn more about T-score and Z-score here.
  5. Reject or do not reject the null hypothesis.

Thanks for Reading!

If you like my work and want to support me…

  1. The BEST way to support me is by following me on Medium here.
  2. Be one of the FIRST to follow me on Twitter here. I’ll be posting lots of updates and interesting stuff here!
  3. Also, be one of the FIRST to subscribe to my new YouTube channel here!
  4. Follow me on LinkedIn here.
  5. Sign up on my email list here.
  6. Check out my website, terenceshin.com.

Related Articles