How to Deflate Your Time Series
We’ll look at what inflation adjustment is and why you should deflate your time series. Along the way we’ll get to know the Consumer Price Index (CPI) and how it is calculated.
Inflation adjustment or deflation is the process of removing the effect of price inflation from data. It makes sense to adjust only data that is currency denominated in this way. Examples of such data are weekly wages, the interest rate on your deposits, or the price of a 5 lb bag of Red Delicious apples in Seattle. If you are dealing with a currency denominated time series, deflating it will extinguish the fraction of the up-down movement in it that was a consequence of general inflationary pressure.
Before we get into the ‘How’ of inflation adjustment, let’s look at the effect that inflation adjustment can have.
The effects of inflation adjustment
The time series below represents the average yearly salary of all wage and salary earners in the United States from 1997 to 2017. The data shows a modest average year-to-year growth of roughly 3%.
When you adjust this data for inflation, the graph turns decidedly choppy:
What did we do to get this second graph? What we did was to take the Consumer Price Index, specifically the CPI-Urban Wage Earners and Clerical Workers: All Items 1982=100 published by the U.S. Bureau of Labor Statistics and we divided the yearly salary data with the CPI value for that year and multiplied the result by 100. For example for 1997, we divided the wage $43615 by the CPI for 1997 which was 157.6 and multiplied the result by 100, to get the inflation adjusted wage (in 1982 Dollars) of 43617/157.6*100 = $27674. This calculation was then repeated for each year to get the second plot shown above.
Here is another plot that illustrates the effects of inflation adjustment.
The blue line in the plot shows the percentage change in wages from year to year before deflation, the orange line shows the percentage change in ‘real wages’ after deflation, and the gray line shows the inflation for each year.
For the most part, the changes in the inflation-adjusted wages have tracked the changes in the non-adjusted wages. But in years with low inflation as compared to previous years (e.g. 2009 & 2015), the inflation adjusted wages have risen sharply, while in years where inflation has risen a lot as compared to the previous year (e.g. 2000 , 2008 and 2011), the inflation adjusted wages have taken a nose dive.
Now that we have seen a couple of examples of how inflation adjusted data looks like, let’s get down to the nuts and bolts of how inflation adjustment works.
The formula for inflation adjustment
As we have seen, you can adjust for inflation by dividing the data by an appropriate Consumer Price Index and multiplying the result by 100.
This is an important formula. Let’s tag it as Equation I. We’ll need to use it again soon.
There are two things you should know while using this formula:
- In the denominator of this formula, it is important to use the correct price index — there are usually several price indexes that you can choose from, and
- It is important to know how to interpret the inflation adjusted value. How you interpret this value depends on what data you have and which price index you have used to deflate it.
Let’s inspect each one of these points in detail.
Which Inflation index should I use?
There are usually several kinds of CPI available and you should use the right kind for your category of data. For example the US Bureau of Labor Statistics (BLS) publishes a large number of price indexes. Following is a sample set:
Which index you use depends on what data you wish to deflate and what property of your data you wish to measure. Let me illustrate this point with two examples:
Example 1: Suppose you have a time series for the average yearly apple prices found across all urban areas of United States. If you wish to bring out the core-growth in the price of apples experienced by urban consumers in the US, after discounting the effect of the overall inflation experienced by urban consumers, you should use the CPI-All Urban Consumers: U.S. All items, 1982–84=100 or the CPI-All Urban Consumers: U.S. All items, 1967=100 to deflate your apple price data. If you make the mistake of using the very popular index CPI-Urban Wage Earners and Clerical Workers you will get incorrect results because this index measures the price inflation experienced by only urban wage & salary earners, not by all urban consumers.
Example 2: Say you wish to bring out the core growth in apple prices after cancelling out the effect of overall urban food inflation, you should use the index CPI-All Urban Consumers: Food and beverages in U.S. city average, all urban consumers, not seasonally adjusted to deflate your data. This will give you a measure of how much dearer or cheaper apples became w.r.t. to other food items and only for urban consumers. I’ll describe this particular use of CPI in more detail later.
Now that we have understood the importance of using the correct index and how to use it to get the adjusted values, let’s drill deeper into what the adjusted value is telling us. For this we’ll first need to understand the concept of CPI.
How to interpret the inflation adjusted values?
To know how to interpret the deflated values, one must understand what CPI is and how it is calculated. The technical definition of CPI sounds boring but here it is anyway:
The Consumer Price Index measures the change in the price of a market basket of goods and services consumed by a household.
To really understand what CPI is one must know how to calculate it, and this calculation is best illustrated by an example. So, let’s launch a mini-project to create a shiny new index. We’ll call it CPI Fictitious in Gotham City — All Items, not seasonally adjusted.
Our first task will be to calculate the yearly expenses of a fictional household in Gotham City.
We’ll also make a rather daring assumption: we’ll assume that our fictitious household’s consumption perfectly represents the consumption of all households in Gotham City. Armed with this assumption, let’s get down to the task of computing the household’s expenses.
The following table contains the breakup of the household’s yearly expenses among 8 categories measured for two consecutive years:
In this example, the market basket is the collection of goods and services that our fictional household consumes every year. For the fictional household in Gotham, the price of the market basket was $42000 in 2018 and it was $43260 in 2019, an increase of 3% over 2018 which we will attribute to inflation.
Notice that the market basket has been spread across eight categories. These are also some of the categories that the US Bureau of Labor Standards uses while calculating the various kinds of CPIs in the United States.
Now let’s recollect that we want to arrive at is an index and not the absolute dollar cost of the market basket. An index needs a base with which all future values can be easily compared. So we will arbitrarily assume that 2018 is our base year and we’ll set the value of the index in 2018 to 100 points. Let’s bring this out in the name of our index by renaming it to:
CPI Fictitious in Gotham City— All Items, not seasonally adjusted, 2018=100.
We can now calculate the value of CPI Fictitious in 2019 as follows:
Thus, value of index in 2019 = $43260/$42000 * 100 = 103.
What the CPI value of 103 is telling us is that the market basket became 103/100 = 1.03 times more expensive in 2019 for all households in Gotham.
In general:
By time period (or simply period), we normally mean a specific day, week, month, quarter or year.
Suppose the price of the market basket was measured in 4 consecutive years : 2020 to 2023 and was found to be
$44400 in 2020,
$46200 in 2021,
$43800 in 2022, and
$45240 in 2023.
Then our made-up index takes on the following values in those years:
In 2020, index value = $44400/$42000 * 100 = 105.7143
In 2021, index value = $46200/$42000 * 100 = 110
In 2022, index value = $43800/$42000 * 100 = 104.2857
In 2023, index value = $45240/$42000 * 100 = 107.7143
Following is the resulting table of values:
Once you have the CPI data, calculating year-to-year inflation is very easy. Here is the formula:
Using this formula, we can see that Gotham City households experienced the following inflation from 2019 to 2023.
Now let’s look at the effect of inflation on just one of the items in the market basket — say a bowl of mulligatawny soup .
During the time frame 2018–2023, suppose mulligatawny soup showed the following price trend:
Clearly some of the inflation in the price of soup was because of the general inflationary pressure in Gotham’s economy during those years. We saw that Gotham experienced an overall inflation of 3% in 2019. We’ll assume that the cost of mulligatawny soup must have also increased by at least that much in 2019.
Let $X be the cost of soup in 2018. If after 3% inflation the cost became $3.26 in 2019, then $X * 103/100 = $3.26. So the cost of soup in 2018 ought to have been:
$3.26*100/103 = $3.16505
So $3.16505 is the cost that soup ought to have had in 2018 for it to have cost $3.26 in 2019 after a 3% price rise. Another way of interpreting the value of $3.16505 is to say that this is how much soup cost in 2019, but in 2018 dollars.
We know from the table that $3.08 is how much soup cost in 2018 in 2018 dollars. So now we can do an apples-to-apples comparison of two costs: $3.08 and $3.16505 because both of them are expressed in 2018 dollars.
We can now calculate the intrinsic inflation in the cost of soup in Gotham in 2019 after discounting the overall inflation from 2018 to 2019 as follows:
Similarly, the price of soup in 2020 can be adjusted to 2018 dollars by dividing it by the amount of inflation that Gotham experienced from 2018 to 2020.
To calculate this inflation amount, recollect equation (III): the formula for inflation. We’ll reproduce it below:
We can re-purpose this formula to find the inflation rate in the current time period as compared to the base year (2018). Let’s work through the math:
Remember we said we’ll revisit Equation (I)? Here it is again:
We can express Equation (IV) in terms of (I) by expressing the denominator of Equation (IV) as follows:
Now referring back to Equation I, CPI for 2020 is the index value and CPI for 2018 is always 100 since its the base year. Therefore,
Which is the same as what we got before using a different but equivalent formula.
Either way, we now have the cost of soup for both 2019 and 2020 adjusted to the same base i.e. 2018 dollars. Therefore like we did before, let’s compare the real percentage change in the price of soup from 2019 to 2020 after discounting the effect of overall price inflation. This ‘core’ change in price is:
We can now state the general formula for calculating the real (intrinsic) change in the price of an item with respect to a CPI of interest:
The following table shows the inflation in soup price in Gotham in the years 2019 through 2023 before and after adjusting for overall inflation.
That was a healthy dose of math. Let’s take a break and enjoy some soup before ploughing ahead.
Our fictitious CPI example has served us well. So far it showed us how to calculate the index, how to calculate inflation in the index, how to interpret inflation adjusted values and how to calculate the intrinsic growth of an item’s price w.r.t. an appropriate index.
But now, at the risk of deflating the interest (and egos) of all us Batman fans, we must return to reality.
A quick dose of reality
Each year the BLS in the US, (or the corresponding government agency in other countries) interviews thousands of households across different financial strata (low, middle, high income), and different professions, in order to gather data about their spending habits. The result of this exercise is the calculation of dozens of smaller, more focused indexes. These indexes are then aggregated to form several top-level CPIs using a system of weights. The exact calculations can get complicated, but the concept is simple. You want to find out how much money the ‘average’ household spends on goods and services and then index that amount to some arbitrary base year. Then repeat this calculation each month, quarter and year to get a sense for how much more expensive (or cheap) the cost of living has become as compared to the previous time period.
If you are training a model on currency denominated data…
If you are training a model, your training will not fail if you do not deflate your currency denominated data. At the same time, you should consider deflating it because doing so will remove the portion of the ‘signal’ from your data that is due to general inflationary pressure. Along with deflation, you should also consider one or more of other transformations such as the log transformation (which makes the trend linear), seasonal adjustment, and differencing. All these operations will remove the corresponding portions of the signal from your data. What will remain is the residual trend that your training algorithm can now focus on. Plus of course there will be the noise —which, your algorithm will have to learn to ignore!
Happy deflating!
I write about topics in data science, with a specific focus on time series analysis and forecasting.
If you liked this article, please follow me at Sachin Date to receive tips, how-tos and programming advice, on topics devoted to time series analysis, modeling and forecasting.
