Hands-on Tutorials
Multivariate Autoregressive Models and Impulse Response Analysis
Econometric-focused introduction to vector autoregression models and impulse response functions for multivariate macroeconomic analysis.
Consider the difficulty of discovering meaningful patterns in time; for instance, with the evolution of sales, the change in weather, or even one’s own sleep cycle. Compared to other data types, time series data has its own unique considerations. To disambiguate, the stochastic yet cyclical qualities require modeling techniques particularly suited to pattern recognition over time. Single variable or univariate time series models are straightforward and effective; however, a model with a single dependent variable can be an oversimplification, requiring strong assumptions that may not be feasible in real-world scenarios. This is particularly relevant in macroeconomics, which almost always requires modeling multiple dependent variables. Hence, macroeconomics relies heavily on econometric models developed specifically for multivariate time series data.
The use of multivariate autoregressive models is not exclusive to economics— they are very useful in business settings, although the primary usage is for the purpose of forecasting. Economists, on the other hand, are also interested in understanding impulse response, which necessitates moving beyond forecasting to assess causal impact. Through my consulting work at the World Bank, I discovered that empirical macroeconomic research requires an understanding of how to translate multivariate autoregressive models into impulse response functions. Therefore, the purpose of this article is to introduce vector autoregression (VAR) models and impulse response analysis. The intent is to provide a starting point for this topic, which can be optionally extended with the offered suggestions of helpful resources for further exploration.
To start, I cover the econometric background needed to contextualize the use of multivariate time series models for macroeconomics. Next, I briefly detail the setup and functionality of a VAR model. Following that, I discuss impulse response analysis, focusing both on the extrapolation of impulse response functions and the assessment of Granger causality. I conclude with practical details concerning the implementation of VAR models in both Python and R, and offer suggestions for the interpretation of the resultant impulse response functions.
Time series Econometrics
Autoregressive is a term that describes a time-varying stochastic process. Accordingly, time series econometrics provides autoregressive statistical models to describe economic time series data. When working with time series data, there are two statistical properties to consider: stationarity and cointegration. Stationarity refers to a property of a time series, such that the mean, variance and autocorrelation structure will remain constant over time. This means that a stationary time series will change over time in a way that remains consistent. To be clear, this does not mean that the series itself remains constant. More explicitly, a stationary time series has constant variance over time, and is without trend or periodic fluctuations, such as those caused by seasonality. More often than not, time series are not stationary, and this requires transforming the series to allow for the assumption of stationarity. Methods of transformation include differencing the data, or taking the logarithm or the square root of a time series. For more details about stationarity in time series, I suggest this helpful Medium post.
Cointegration is a statistical property of a collection of time series, which is particularly relevant in the multivariate setting. Given a set of non-stationary time series modeled as a system of equations, if a stationary linear combination of the system (that has a lower order of integration) exists, then the time series combined (multivariate time series) can be said to be cointegrated. More specifically, since a multivariate model has several variables, it is possible that cointegration relationships exist between any combination of those variables. In a practical sense, this property is important for understanding stochastic or deterministic trends. In a statistical sense, cointegration means that the differences in the means of a set of time series remain constant over time, without offering an indication of directionality. In other words, when a multivariate time series is cointegrated there is a long run, statistically significant relationship between one or more combinations of the variables. For more information about cointegration in time series, check out this short Medium post.
Since country-level economic observations tend to be low frequency, a complication is that macroeconomic data tends to be sparsely available. For example, gross domestic product (GDP) is usually released as an annual figure. Nonetheless, when the objective is to understand the behaviour of an economy as a whole, it is natural to look at time series data. Therefore, macroeconomists have a vested interest in understanding the statistical properties of a time series, properties that change depending on the frequency and duration of the series. As a disclaimer, this introduction is not a comprehensive review of all the important statistical properties of a time series. For example, within macroeconomics, the concepts of conditional heteroskedasticity and stochastic volatility are also important for studying time series. In this article, I focused on stationarity and cointegration, since these two statistical properties are important for understanding the underlying assumptions of vector autoregression (VAR) models, which are described in the following section.
Vector autoregression models
For multiple time series, the econometric model of choice is a vector autoregression (VAR) model. Within the field of machine learning, this model is sometimes referred to as a VAR forecasting algorithm. The basic VAR model can be described as covariance stationary, this means that we are assuming that the time series process is stationary and ergodic with time invariant means, variances, and autocovariances (Zivot & Wang, 2003). A simplification of a VAR model is that it explains the endogenous variables solely by their own history, apart from deterministic regressors (Pfaff, 2008). What makes a model autoregressive is that each time series is modeled as a function of the past values, which means that the predictors are time delayed values of the series, also known as “lags”. In a univariate autoregressive model, the predictors or lags influence the dependent variable but not vice versa; conversely, in a multivariate model like VAR, the variables (ie. time series) influence each other in a bi-directional relationship.
Since the multiple time series influence each other, they can be modeled as a system of equations with one equation per time series (variable). Simply put, each time series in a VAR, is modeled as a linear combination of past values of itself and the past values of other time series in the system. Mathematically, the VAR process is described in the following equation:
Formally, the VAR process models a T * K multivariate time series Yₜ , where T denotes the number of observations and K the number of variables; where uₜ∼N(0,σᵤ) and Aᵢ is a K * K coefficient matrix (statsmodels documentation). Here, the word “variable” is used interchangeably to refer to an individual time series, so K is the number of single time series while Yₜ is a collection of the series. The number of lags ranges from 1 to p, where the lags are used as the predictors in the above equation, and the value of p represents the order of the system of equations. The beta values represent the coefficients of the lags of Yₜ , and range from β₁ to βₚ. The last term “uₜ” is the error term and can be thought of as “white noise”.
Consider the equation below, it shows the simplest scenario where we have a single lag and two time series: Y₁ and Y₂. The goal is to forecast these series at time t; therefore, calculating Y₁,ₜ requires the past values of both Y₁ and Y₂. Similarly, calculating Y₂,ₜ requires the past values of both series.
In the system of equations above, Y₁,ₜ -₁ is the first lag of Y₁ and Y₂,ₜ -₁ is the first lag of Y₂; both of these first lags are included in each equation. Since there is only one lag for each series, this means that the equations are of order 1 (ie. p is 1). When p is order 1, the VAR model itself can be described as a first order model. For an excellent overview of the intuition behind the VAR process, I recommend this blog post by Selva Prabhakaran.
When building a VAR model, the lag order p is very important and is determined both by the number of observations, T and the number of equations, K. Determining the optimal lag order p, in practice, is accomplished by iteratively fitting the model with an increasing lag order and selecting the model with lowest Akaike information criterion (AIC) score. There are other metrics that can be used for model selection, but the AIC score is considered best practice. For more information, I suggest this Medium post that covers some details of the AIC model selection criterion. For a deeper understanding, I suggest Ritei Shibata’s paper on the topic (Shibata, 1976).
There are several variations of the basic VAR model, each has specific alterations that are intended to overcome the drawbacks of the basic model. In the absence of exogenous variables (X), the disturbance variance-covariance matrix contains all relevant information about contemporaneous correlation among the variables in Yₜ (Baum, 2013). In this case, it is possible to use a structural VAR (SVAR) model; it is useful for when theory can be used to place restrictions on the contemporaneous correlations. Alternatively, if the variables in Yₜ are not covariance stationary, but their first differences are, they can be modeled with a vector error correction model (VECM). This is because equilibrium relations exist between the variables, which can render them stationary without taking differences. That is to say, a VECM is suitable when cointegration exists between variables, because then it is possible to work with a non-stationary (ie. not covariance stationary) multivariate time series by incorporating the cointegration relationships (Baum, 2013).
Essentially, the choice of model depends on the depth of domain knowledge available to make subjective restriction decisions, and the statistical properties of the time series. Despite my instinct for simplicity, I discovered that there are practical reasons to move beyond the basic VAR model. Perhaps even to the point of considering other macroeconomic approaches such as narrative methods or Cholesky decompositions. For example, failing to account for anticipated movements in government spending, will lead to VAR results that are inconsistent with a narrative approach (Ramey, 2011).
Impulse response analysis
Although VAR models are also used in the field of machine learning, the primary usage is for forecasting. Differentiating economics, is the use of VAR models for analyzing impulse response and assessing Granger causality. An impulse response can be defined as the reaction of a system, in response to an external change. In the macroeconomic context, the system is a system of equations that comprise a multivariate autoregressive model, and the external change is referred to as an exogenous shock. All the variables in a VAR model depend on each other; therefore, individual coefficient estimates provide limited information on the reaction of the system to a shock (Zivot & Wang, 2003). Hence, the need for impulse responses, which provide information about the dynamic behaviour of a VAR model. Subsequently, the purpose of an impulse response function (IRF) is to describe the evolution of a VAR model’s reaction to a shock in one or more variables (Lütkepohl, 2010). In other words, an IRF allows us to trace the transmission of a single shock within a system of equations, a feature which makes them useful for evaluating economic policy.
To calculate the IRF, we rewrite the VAR equation described earlier, in its moving average form. This is shown in the equation below:
This form is the vector moving average representation of the VAR model, in which all the past values of Yₜ have been substituted out. Importantly, taking the moving average form smooths out short term fluctuations and highlights longer term trends. In the above equation, X represents an optional set of exogenous variables and the Dᵢ matrix denotes the dynamic multiplier functions, which are also known as transfer functions. The symbol𝚽 represents the moving average coefficient, the sequence of which “𝚽ᵢ”, represents the impulse response functions at horizon i. For a deeper understanding of impulse response analysis, I suggest an article from the Journal of Econometrics (Koop et al., 1996) and another from Economic Letters (Pesaran & Shin, 1998).
As mentioned earlier, a drawback of the basic VAR model is the inability to describe contemporaneous relationships between the variables. With impulse response analysis, if there is a need to describe those relationships, it is important to know the contemporaneous effects of a shock to the multivariate system. Generally, this is accomplished by arranging the variables of the model in a suitable order by generating orthogonal impulse response functions. More specifically, orthogonality is applied where the correlation between the errors is obtained from the lower Cholesky decomposition of the error covariance matrix. The alternative is to use a SVAR model where subjective restrictions are used to model the relationship between contemporaneous variables directly.
Causality is very important within economics, and for time series data, Granger causality is the statistical test used to determine a specific type of causality. Depending on the philosophical definition of causality that one holds, one might argue that Granger causality is not true causality, but merely “predictive” causality. A Granger causality test specifically measures a time series’ prediction ability of future values using prior values of another time series. To be precise, Granger causality tests for temporal relations in the context of whether one variable forecasts another (Granger, 1969). As a caveat, econometricians refer to the post hoc fallacy, where it is a fallacy to assume that simply because something preceded another, that this is sufficient proof of causation. Regarding VAR models, the most that Granger causality allows, is the claim that a set of variables are Granger-causal within one of the VAR equations. For a short tutorial on testing for Granger causality, I suggest this Medium post. For a more in-depth understanding, I recommend the “Time Series Analysis” textbook by James D. Hamilton (1994).
A further note about causality, Professor Valerie A. Ramey has created an excellent and accessible overview of causality in macroeconomics, I highly recommend her slide deck on the topic. She mentions that identification is key to turning correlation into causal relationships, a fact that is complicated by the need to address the issues of dynamics, general equilibrium effects and expectations. When it comes to impulse response analysis, other than the basic VAR, there are two common methods that are worth exploring: dynamic simulation (Romer & Romer, 2010) and Jorda local projections (Jorda, 2005). To better understand the application of impulse response in macroeconomics, I suggest two of Ramey’s papers that can provide context for the usefulness of impulse response analysis: “Government Spending Multipliers in Good Times and in Bad: Evidence from U.S. Historical Data” (Ramey, 2014) and the more recent “The Macroeconomic Consequences of Infrastructure Investment” (Ramey, 2020).
Model implementation and interpretation of the IRF
I have found that the best source for macroeconomic data is the International Monetary Fund (IMF). This organization shares data that is free to download from their website, and they offer a RESTful API that can be accessed with Python or R. For Python, they suggest this guide, and for R they offer a package “imfr” that makes it simple to query the API. Personally, I prefer to use the R package as it is the most accessible for data science. In Python, the statsmodels library can be used to build a VAR model, and run a number of statistical tests. In R, the most compressive package is the vars library.
Prior to building a VAR model it is helpful to test individual time series for stationarity and test the multivariate time series for cointegration. There are several unit-root tests available for testing stationarity, such as the Augmented Dickey-Fuller test, the KPSS test and the Phillips–Perron test. The most common test for cointegration is the Johansen test which allows for more than one cointegration relationship in a multivariate time series. A crucial part of building a VAR model is deciding the lag order by optimizing for a model selection metric like the Akaike information criterion (AIC). Fortunately, model selection is easily accomplished with the Python library statsmodels that has a built-in function to select lag order; this determines the optimal lag order based on the best scores of a selected metric.
A visualization is necessary to interpret the impulse response function (IRF), conveniently, statsmodels makes it simple to plot the IRF for a specified lag length. When plotted, as seen in the image below, the asymptotic standard errors are plotted by default at the 95% significance level.
In the image above, the IRFs are plotted for 10 lags, and in each plot the blue line represents the IRF and the black lines represent the confidence intervals. With statsmodels, it is also possible to plot the IRF in either orthogonalized or non-orthogonalized form as required. To test for forecast errors in the impulse response function, we can use forecast error variance decomposition (FEVD). This variance decomposition indicates the amount of information each variable contributes to the other variables in the autoregression. Essentially, this decomposition determines how much of the forecast error variance of each of the variables can be explained by exogenous shocks to the other variables (Pesaran & Shin, 1998).
Final thoughts
This article outlined a personal journey, undertaken for comprehension of experimental results from policy-focused empirical macroeconomic research. Time series econometrics is a fascinating subject, and I am particularly appreciative of the ease of application afforded by the Python package, statsmodels and the R package, vars. The IMF data API and these two packages made it simple to play around with macroeconomic data, such that I was able to build an intuition for the methods prior to diving deeper into theory. The accessibility of data science tools has influenced the speed at which I have been able to learn econometrics. Therefore, I am grateful for the open-source community that creates and maintains such tools.
I hope that this article was a helpful introduction to the topic of multivariate autoregression models and impulse response analysis. As always, I welcome feedback and questions, please feel free to connect with me on Linkedin.
