Time-series Forecasting Flow

A brief introduction on critical steps in demand forecasting

Collecting

The key here is the format of data storage.

Intuitively we think of time-series data in the format below, also known as the wide format.

However, the wide format is bad for SQL-based storage, as when we add new dates, we need to add another column to the table, and thus, best practice is storing the data in long format, as shown below.

Another thing worth mentioning here is that since it is highly likely we are forecasting multiple products here, a convenient way is to use Python dictionary, where key is product ID, and value is the long data format (basically a dataframe). So we end up with a dictionary of dataframes.

Cleaning

Cleaning here is pretty standard. Basically we look for outliers. Common practice is that if we have weird values such as a fixed negative number for demand (which it usually is greater than or equal to 0), we drop them. Another thing we need to look for here is missing values. We could use pandas fillna function, in which we specify if we want to replace them with 0’s or with a method such as the last demand (backfill).

Testing

Also known as back-testing. Basically two most common ways of windowing, one is sliding (training and forecast time are fixed), and the other one is expanding (training data length increased by x at a time). Uber does a great job visualizing the two methods as below.

Common evaluation metrics include MAPE, wMAPE, sMAPE, as well as RMSE and RMLSE (if the data distribution is skewed).

Modeling

The most important take-away here is that often simple methods such as mean of all data points outperform more sophisticated ARIMA. Also FB’s prophet package (in both R and Python) could be a go-to algorithm as well.

Evaluating

When evaluating performance, we could try different training and forecasting length to better understand our data, as well as the algorithms used. For example, Naive method performs extremely well when training data is 1 week and forecasting data is the next week; and ARIMA shines when we have longer data length, in terms of both training and forecasting. In other words, we need to determine our business use case first, whether it is short-term forecast or long-term forecast, so that we can better evaluate the results from different algorithms.

Questions?

jchen6912@gmail.com