The world’s leading publication for data science, AI, and ML professionals.

P-Value Method for Hypothesis Testing

The P-Value method is a method that is commonly used in Hypothesis Testing to validate the Hypothesis. In this article, I will be…

Photo by Chris Liverani on Unsplash
Photo by Chris Liverani on Unsplash

Why Hypothesis Testing is done

Usually, we get Sample Datasets to work on and perform data analysis and visualization and find insights. Through that analysis, we make inferences on the whole population. The population is the entire dataset, whereas Sample Datasets are chunks from the Population Dataset. Similar to population, these sample datasets also have some mean and standard deviation associated.

These samples mean values usually vary from the population mean by different ranges. These ranges can deviate from very close to very far from the population mean. For making correct inferences, our sample datasets need to resemble the properties of the population dataset. So, these samples and the sample means are validated through various Hypothesis Testing methods like the P-value method, Critical Value method, T-test, ANOVA test, etc.

For example, consider a store that sells electronic devices. They launched a new laptop a while back and held a marketing campaign to promote the product. The campaign ran for 100 days. They claim that, on average, 500 of those laptops were sold per day across all branches during the campaign. This claim value (500) is the population mean, which says about the laptops sold per day in that campaign of 100 days. Now, we take out multiple samples, each of which has 20 days of data, and check the mean laptop sales of each sample. We know that each sample will have a mean value that might vary from other samples’ mean. These sample means can differ from the population mean (500). If the sample mean value is close to 500, such as if we get the sample mean of laptops sold as 490 or 498 or 505, we are more likely to believe the claim made by the store. But if the sample means are extreme values, such as 200 or 150 or 710, we are less likely to believe the claim. In such cases, the p-value method helps in validating the hypothesis statistically.


What is the P-value method in Hypothesis Testing?The P-value method is used in Hypothesis Testing to check the significance of the given Null Hypothesis. Then, deciding to reject or support it is based upon the specified significance level or threshold.

A P-value is calculated in this method which is a test statistic. This statistic can give us the probability of finding a value (Sample Mean) that is as far away as the population mean.

The P in P-value stands for Probability.

Based on that probability and a significance level, we Reject or Fail to Reject the Null Hypothesis.

Generally, the lower the p-value, the higher the chances are for Rejecting the Null Hypothesis and vice versa.

Also, we make use of the Z-table to perform this process.


How to perform hypothesis testing through p-value method ?

To perform this method, we first formulate the Null and Alternate Hypotheses.

Then, we have the following given inputs:

Population Mean (X)
Population Standard Deviation (SD)
Alpha or the Significance Level
Sample Size (n)
Sample Mean (x)

Alpha or the Significance level is the probability of making an error in Hypothesis Testing when the Null Hypothesis is true. It also specifies the critical region or the rejection region of the Null Hypothesis in the distribution. If the Sample Mean lies in that critical region, we reject Null Hypothesis and accept the Alternate Hypothesis.

The commonly used alpha is 0.05 (5 %), but it can vary depending on the business requirements. If the test is a 2-tailed test, we divide the alpha by 2 to equally distribute the significance level on the lower and upper cut-off. In the case of a 1-tailed test, we keep the alpha as it is.

You can read more on the two types of tests here.

Then, we calculate the Standard Error of the sample, which is the Population Standard Deviation divided by the square root of sample size (n).

Formula for Calculating Standard Error
Formula for Calculating Standard Error

After finding the Standard Error, we take a sample and the mean of that sample and then find the Z-score associated with that mean value.

Z = (Sample Mean - Population Mean)/ (Standard Error)
Formula for Calculating Z Score
Formula for Calculating Z Score

Now, we find the p-value associated with the Z-score we calculated. Then we refer the Z-table to find the probability of the Z-score calculated. Then, to find the p-value, we subtract that probability from 1.

P-Value = 1 - Probability(Z-score)

Finally, we check if the calculated p-value is greater than the significance level or not.

If the P-value > Significance Level, then we Fail To Reject The Null Hypothesis. Or else, if the P-value < Significance Level, we Reject the Null Hypothesis.


Let’s understand the method better through an example

Let’s take the previous example of the electronic store. The store launched a new laptop a while back. They had run a marketing campaign for 100 days to promote the product. The store claimed that the average number of laptops sold was 500 units per day during the 100 days of the marketing campaign. It means that, on average, the store sold 500 laptop units per day during the campaign across all of their stores. So, the population mean is 500.

So, the Null and Alternate hypotheses would be :

Null Hypothesis (H0) : The Average laptop units sold during the campaign were 500 (Average Laptop units sold = 500)
Alternate Hypothesis (H1) : The Average laptop units sold during the campaign were not 500 (Average Laptop units sold != 500)

So, looking at the Null and the Alternate Hypothesis, we notice that this is a 2-tailed test.

Now, we take around 5 sample datasets from the entire dataset of the campaign. Each sample has data of 15 days (Also, the data in these samples might overlap).

We observe the average of the laptop units sold in each of the samples. We obtain the mean of the 5 samples as follows:

Sample_1 : 535.5
Sample_2 : 495.2
Sample_3 : 510.5
Sample_4 : 497.7
Sample_5 : 504.3

Now, let’s validate Sample 1 using the p-value method. We will take the threshold or the significance level of 0.05 (5%) to validate this hypothesis. Because this is a 2-tailed test, we divide the significance level by 2 for distributing equally on the Upper and Lower critical points.

We assume that the Standard deviation of the population is 45. But the Standard Deviation of the Population is usually known.

We know the condition that if the calculated p-value for any sample is higher than the significance level, we Fail to Reject the Null Hypothesis. Else, we Reject the Null Hypothesis.

Here, we are validating for the 1st Sample (Sample_1).

So, the given inputs we have are as follows:

Population Mean               : 500
Population Standard Deviation : 45
Sample Mean (Sample_1)        : 535.5
Sample Size (Sample_1)        : 15
Alpha (Significance Level)    : 0.05 (2-tailed Test) 

We now calculate the Standard Error

Calculating Standard Error
Calculating Standard Error

The 2nd step is to calculate the Z-score

Calculating Z-score
Calculating Z-score

We got the Z-Score as 3.05. Now, we will refer the Z-table for finding the probability of the Z-score obtained.

Z-table
Z-table

The Z-Score we got was 3.05. Now, in the Z-table, we need to go look for the value in the first column vertically (+3). After that, we need to look for the next decimal value horizontally (0.05). So, we find the probability of the Z-score by going at +3 vertically and 0.05 horizontally. The probability of Z-score comes out to be 0.99886

Now, to calculate the p-value :

P-value = 1 - Prob(Z-score)
P-value = 1 - 0.99886
P-value = 0.001

Since, this is a 2-tailed test, we will multiply the the p-value by 2

P-value = 0.001 * 2
P-value = 0.002

We got the P-value = 0.002

Finally, we will compare this now with the significance level

P-value = 0.002
Alpha (Significance Level) = 0.05

We notice that, P-value < Significance Level

This means that, in this problem statement and the hypothesis, finding a sample mean value of 535.5 has a probability of 0.002 or 0.2%, which is extremely less than the significance level (0.05 or 5%) and is considered too far away from the population mean (500).

Thus, we Reject The Null Hypothesis.

This also means that our sample mean value of 535.5 lies outside the acceptance region and we cannot expect such variance in the value of claim made by the store (Average Laptops sold = 500 per day).

The sample mean value of 535.5 is thus not an accepted mean value for this hypothesis and thus the status quo (Null Hypothesis) needs to be verified and changed.

Similarly, the same process is repeated for all the samples with their sample means and their p-value is calculated and compared with the significance level.

In this way, a hypothesis is validated using the P-value method approach.


Conclusion

In this article, I explained how Hypothesis Testing is done using one of the most common methods called as P-value method by solving a practical example. I would be more than happy to hear your comments and reviews on this article here in the comments. Also, feel free to message me on LinkedIn.


Related Articles