Test for existence of a Trend in a Time Series

Relative Order Test for testing the existence of a Trend in a Time series

Time passes faster for your face than for your feet (assuming you’re standing up). Einstein’s theory of relativity dictates that the closer you are to the center of the Earth, the slower time goes — and this has been measured. At the top of Mount Everest, a year would be about 15 microseconds shorter than at sea level. (Photo by Nathan Dumlao on Unsplash)

A time series comprises four major components. A trend. A seasonal component. A cyclic component. And a stochastic/ random component.

You can have a recap of all the basics of a time series from my following article.

First Steps in Time Series Analysis

We extract all these components and analyze them to get information from a time series. There are lots of standard methods to extract the components from a time series.

But all these components may air may not be present in a time serious altogether. Therefore, before estimating these components, we need to first check for their existence. If they are present then we can move forward with their estimation.

This article explains the Relative Order Test for testing the existence of a trend.

Let's start with an example. Suppose we have 12 observations of a time series. And, we need to find whether a trend is present or not.

Fig.1 Example

Intuition

Once you skim through the observations, it will be evident that a trend exists. The values gradually move from 1 to 11.

From Fig.2 also, we can say, a rising trend exists.

Fig.2 Time plot

Our intuition says that the trend exists, now lets us try to prove this mathematically.

Kendall’s Tau

It is a non-parametric measure of a relationship between columns of sequential data. And time series is sequential. Hence we can use Tau to check the relationship between time and variable Y.

If they are highly correlated, then we can say a trend exists, as the timestamps are always increasing.

Consequently, if they are positively correlated, an increasing trend exists. Whereas, if they are negatively correlated, a decreasing trend exists.

Kendall’s Tau is given in Fig.3

Fig.3 Kendall’s Tau

In Fig.3, T is Kendall’s Tau, C is Concordant pairs, and D is Discordant pairs.

The pairs (i, Y(i)) and (j, Y(j)), where i and j are the timestamps, and Y(i) and Y(j) are the value of the variable at that time, forms a Concordant Pair if:

Fig.4 Concordant Pairs

They form a Discordant pair if:

Fig.5 Discordant Pairs

If T=0 or the number of concordant and discordant pairs are equal, no trend exists. Because, if C = D, the occurrence of a rise or a fall is equally likely. Hence we cannot say a trend component exists.

If T=1 or T=-1, a strong trend exists. Because if C=0 or D=0, the series have either a rising trend or a falling trend. In short, a trend exists.

Now, let us calculate the concordant and discordant pairs for the example given.

Fig.6 Calculating C and D for the given example.

We calculate the value of T=0.85 from the formula in Fig.3. That is close to unity. Hence, we can say that a strong trend exists, which is also coherent with our intuition.

But we have to explain what “close” means in the above sentence. For that, we will perform some statistical tests.

Statistical testing

Once we have calculated T, we can calculate the z-score and p-value. For a 95% confidence interval if p-value<0.05, we can reject the null hypothesis. Which states that there is no trend present.

The formulas for z-score and degree of freedom are given in Fig.7.

Fig.7 Formulas for calculating Z-score and dof

So let's apply this statistical testing to the example given.

We define the hypotheses as:
Null Hypothesis H0: No trend exists
Alternate Hypothesis Ha: Some trend exists

Also, previously we have calculated the following:
T = 0.85
N=12
Degree of Freedom (dof) = N-2 = 10

We can put the values in Fig.7, and get z = 3.85 and dof=10.

Corresponding to these values, we arrive at p-value = 0.001, which is less than 0.05.

Hence we can reject the null hypothesis. Therefore, we can say, there is a trend present.

Conclusion

We have mathematically tested our intuition. And, it comes that it was right. This is called Relative Order testing, which is a very easy technique to check the existence of a trend in a Time series

These statistical test also tells us how strong the trend is! As we are finding a relationship between timestamps and the value of the variable. Since timestamps are always strictly increasing, this correlation helps us to find the strongness of the trend present.

For more such content on machine learning, data science, and programming, please do visit my youtube channel.

Aayush Ostwal

Keep Reading!
Keep Learning!