12 Things You Should Know About Time Series

Published in

Towards Data Science

8 min readMar 22, 2022

For those of you that don’t know, a time series is simply a set of numeric observations which are collected over time (Figure 1). Examples of time series appear in many domains, from retail (e.g. inventory planning) to finance (stock price forecasting). Time series are interesting because of their underlying uncertainty — data changes over time which makes it difficult to understand their future behaviour.

Here I’ll describe 12 properties or components which are important to consider when modelling a time series. In some way or another, these characteristics influence decisions in the data science pipeline for the development of forecasting systems.

Here’s a one liner for each topic if you’re in a hurry:

Trend: long-term change in the mean of the data;
Seasonality: regular and predictable changes;
Residuals: de-seasonalised and de-trended series;
Stationarity: When time series properties remain constant over time;
Autocorrelation: Correlation with past observations;
Heteroskedasticity: Changes in the variance;
Regularity: Whether the series is captured at regular intervals;
Frequency: Frequency at which the series is observed;
Reflexivity: When the forecast affects the outcome;
Outliers: Rare but possibly interesting observations;
Regimes and Change Detection: When the data distribution changes;
Dimensionality: Number of variables in the time series.

1. Trend

Trend is one of the basic building blocks of a time series. It represents the long-term change in the mean of the data as can be seen in Figure 1. This graphic shows an example time series which represents the monthly number of passengers in an airline over time. The mean level of the time series increases over time, which represents a clear upward trend.

Some learning algorithms struggle to cope with the trend component of time series. So, it is often recommended to remove it for an optimal modelling of the time series. You can do this using a differencing operation. Differencing simply means taking the difference between the value of the current observation and the value of the previous one. Figure 2 shows the air passengers time series after removing the trend by differencing; the mean level of the series becomes stable after this process.

Figure 2: De-trended time series after applying a differencing operation.

2. Seasonality and Cyclic Patterns

If a time series experiences regular and predictable changes in fixed periods (e.g. every month) it has a seasonal component. The air passengers time series shows a monthly seasonality, which is apparent by the recurrent oscillations.

Similarly to the trend, the seasonal component also breaks stationarity and it is often recommended to remove it. One can do so also by differencing, but instead of subtracting the previous value from the value of the current observation we subtract the value of the previous observation from the same season.

Seasonal differencing mitigates predictable fluctuations, which also stabilises the mean level of the series. After removing the seasonal component, the time series is referred to as seasonal adjusted.

Besides seasonal effects, time series can be characterised by other predictable oscillations which do not have a fixed period. This type of variation is a cyclic pattern. The typical example of a cyclic pattern is an economic cycle, in which the economy experiences periods of growth and periods of recession.

3. Residuals

After removing the three above components (trend, seasonality, cyclic patterns) from the time series, the remaining part is known as the irregular component or residuals. An example is shown in Figure 3. Residuals are not explainable by any trend, seasonal or cyclic behaviour, but still can cause an impact on the dynamics of the time series.

At any given point in time, a time series can be decomposed in an additive fashion into the above-mentioned components as follows:

y = Trend + Seasonal + Cyclic + Residuals

Depending on the data, this decomposition can also be multiplicative by replacing the sum operations with product ones.

4. Stationarity

Components such as trend or seasonality break time series stationarity. A time series is stationary if the properties of a time series do not depend on the time when the data is observed.

More formally, but without going into the nitty gritty details, a time series is considered to be stationary if there are no systematic changes in the mean or variance, and if periodic variations have been removed.

Many time series techniques work under the assumption that the time series is stationary. When it’s not, operations such as differencing are used to make it stationary.

5. Autocorrelation

The notion of time series implies that there is some degree of dependency on historical data — that what we observe today depends on what happened in the past. The autocorrelation of a time series quantifies this dependency according to the correlation of each observation with its past values. This property provides important structural information about the series. If a time series shows a low autocorrelation on all lags it is referred to as white noise.

6. Heteroskedasticity

This is a complicated word, but the concept is actually simple. A time series is said to be heteroskedastic, as opposed to homoskedastic, if its variance is not constant and changes over time. In the air passenger example, it is clear that the variability of the data increases over time. It’s common that this change in variance co-occurs with a change in the mean level of the data — the variance is typically higher for higher mean values. Heteroskedasticity poses a problem in the process of modelling the data, and there are approaches designed to cope with it. Power transformations, such as taking the log or, more generally, Box-Cox transformations, are usually applied to stabilise the variance. Figure 4 shows an example where the Box-Cox method is applied to a time series.

Figure 4: Time series after applying the Box-Cox transformation

7. Regularity and Intermittency

Time series are typically collected at regularly spaced intervals of time, for example, every day or every hour. These are referred to as regular time series, and most time series methods work under a regularity assumption. However, there are many applications where the time series are intrinsically irregular. For example natural disasters (e.g. earthquakes) or the sales of a particular retail product, which occur in irregular time intervals.

Usually, the irregularity of time series is dealt with interpolation, which renders the series regular. For example, a time series relative to the sales of a product can be transformed to a count of sales in some period (e.g. hourly product sales). This interpolation process may lead to a sparse or intermittent time series, in which there are several observations with constant zero as a value (e.g. no product sales in a given hour). This intermittency is a common hurdle in time series forecasting for inventory planning where some products are sold infrequently.

8. Sampling Frequency

The sampling frequency of a time series denotes the regularity in which it is collected, for example, daily or monthly. Time series with distinct frequencies entail different challenges. The seasonal component may be more difficult to capture for higher frequency time series. Daily or sub-daily time series often comprise multiple seasonal patterns which are not trivial to capture.

Working with low frequency time series is simpler in terms of seasonality. However, there may be other issues to consider. Low frequency datasets typically comprise a smaller sample size relative to higher frequency ones. Classical time series models, such as ARIMA or exponential smoothing, can handle this issue well as they have a small number of parameters. Learning algorithms with more parameters may be prone to overfitting.

9. Reflexivity

A time series is reflexive if the predictions affect the unfolding of events. The classical example of a reflexive time series is stock market data. Forecasting an increase of a stock price attracts investors, which creates demand and drives the price of the stock up. The forecast is then self-fulfilling. On the other hand, the prediction of a market crash can itself cause a market collapse due to the panic of investors. There are also self-defeating reflexive systems, in which forecasting a given event makes it less likely.

Reflexivity may lead to unexpected consequences. Practitioners should identify how it could arise in their time series and somehow incorporate the response in their forecasting system.

10. Outliers

Outliers or anomalies are rare events which deviate significantly from other observations. These instances are common to all types of data and not only time series. Still, in time series, outliers pose an additional challenge due to the temporal dependency among observations.

Time series outliers may occur in just a single instance (point outlier), or span several time steps (subsequence outlier). It is often important to take context into account when searching for anomalies. For example, a 0º temperature may be common in the winter, but an anomaly during summer.

The most appropriate way to deal with outliers depends on their nature. An outlier may arise due to erroneous data collection or sensor malfunction. This kind of outliers represent unwanted data which does not follow the distribution generating the observations. However, there are time series outliers which are in themselves events of interest. Examples of this include stock market crashes or fraud detection, where the goal is to anticipate or mitigate the impact of these rare events.

11. Regimes and Change Detection

A change point, also referred to as concept drift, happens when the distribution of the time series changes. Changes can be recurring; a time series may be characterised by different regimes or concepts and the data distribution changes across these regimes. Regime switching models are popular approaches for this type of problem.

Changes can also be permanent. These are known as structural breaks. These changes pose a challenge to learning algorithms, which must be able to detect them and adapt accordingly in a timely manner.

It’s important not to confuse change detection with outlier detection. The first is concerned with the detection of changes in the regimes governing the time series. When regimes change, the distribution of observations changes accordingly. On the other hand, an outlier represents an observation (or subsequence of observations) which deviates significantly from the typical behaviour, where typical behaviour is characterised by the current underlying regime.

12. Dimensionality

The properties listed so far assume that the underlying time series is represented by a single dimension, where dimension denotes the number of variables. Accordingly, these time series are referred to as univariate. Sometimes however, a time series contains additional dimensions and is thus referred to as a multivariate time series. The additional variables in multivariate time series can be used as explanatory variables when modelling a particular target variable of the time series.

Summary

In this post I listed 12 time series properties that are important when building forecasting models. From basic components, such as trend or seasonality, to reflexivity or change detection, these characteristics may significantly influence the performance of predictive models.