# Introduction to gmvarkit

#### 2018-09-22

The package gmvarkit contains tools to estimate and work with Gaussian Mixture Vector Autoregressive (GMVAR) model (see Kalliovirta et. al. 2016). This vignette does not explain details about the model and it’s assumed that the reader is familiar with the cited article(s).

The GMVAR models in gmvarkit are defined as class gmvar objects, whose can be created with the estimation function fitGMVAR or the constructor function GMVAR. The created gmvar objects are then conveniently used as main arguments in many other functions and methods, that allow for example model diagnostics, simulations and forecasting. Therefore after estimating a GMVAR model it’s straightforward and easy to use the other functions for further analyses. However some tasks, such as applying general linear constraints, creating GMVAR models without estimation or setting up initial population for the genetic algorithm employed by the estimation function, require accurate understanding on how the parameter vectors are constructed in this package.

The notations for (non-constrained) parameter vector are in line with the cited article by Kalliovirta et. al. (2016), but for clarity we will repeat these notations in the first section. In the second section we show how to apply general linear constraints to the autoregressive parameters of GMVAR model. Two examples are presented to demonstrate how to use the general formula. In the third section we have listed some useful functions and methods found in gmvarkit and briefly explain how they work and how to use them.

## General notations and regular models

### General notations

Defining a GMVAR model requires specifying the autoregressive degree denoted by symbol p and the number of mixture components M. The number of time series in the system is denoted by d, and it’s assumed that d is larger than one.1 The form of the parameter vector depends on whether a regular or constrained model is considered.

### Parameter vector of non-constrained model

The paramater vector for regular model is size ((M(pd^2+d(d+1)/2+1)-1)x1) and has form $\boldsymbol{\theta} = (\boldsymbol{\upsilon_1},...,\boldsymbol{\upsilon_M},\alpha_1,...,\alpha_{M-1}),\enspace \text{where}\quad\quad\enspace\quad$ $\boldsymbol{\upsilon_m}=(\phi_{m,0},\boldsymbol{\phi_m},\sigma_m)\enspace ((pd^2+d(d+1)/2)x1),\quad\quad\enspace$ $\boldsymbol{\phi_m}=(vec(A_{m,1}),...,vec(A_{m,p}))\enspace (pd^2x1), \enspace \text{and}\quad\enspace\;$ $\sigma_m=vech(\Omega_m)\enspace ((d(d+1)/2)x1),\enspace m=1,...,M.$

Above $$\phi_{m,0}$$ denotes the intercept parameter of m:th mixture component, $$A_{m,i}$$ denotes the coefficient matrix of m:th mixture component and i:th lag, $$\Omega_m$$ denotes the (positive definite) error term covariance matrix and $$\alpha_m$$ denotes the mixing weight parameter of m:th mixture component. $$vec()$$ is a vectorization operator that stacks columns of a matrix into a vector and $$vech()$$ stacks columns of a matrix from the main diagonal downwards (including the main diagonal) into a vector.

The parameter vector above has “intercept” parametrization, referring to the intercept terms $$\phi_{m,0}$$. However it’s also possible to use “mean” parametrization, where the intercept terms are simply replaced by the regimewise means $$\mu_m=(I_d-\sum_{i=1}^pA_{m,i})^{-1}\phi_{m,0}$$.

## Models with linear constraints

### General linear constraints and parameter vector

Employing linear constraints on the autoregressive parameters of GMVAR model is straightforward. The constraints are expressed in a very general form allowing any arbitrary linear constraints, but one needs to take the time to construct the constraint matrix carefully for each particular case.

We consider constraints of form $(\boldsymbol{\phi_1},...,\boldsymbol{\phi_M}) = \boldsymbol{C}\boldsymbol{\psi},\enspace \text{where}\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\enspace$ $\boldsymbol{\phi_m}=(vec(A_{m,1}),...,vec(A_{m,p}))\enspace (pd^2x1), \enspace m=1,...,M,$ $$\boldsymbol{C}$$ is known $$(Mpd^2xq)$$ constraint matrix (of full column rank) and $$\boldsymbol{\psi}$$ is unkown $$(qx1)$$ parameter vector.

The paramater vector for constrained model is size ((M(d+d(d+1)/2+1)+q-1)x1) and has form $\boldsymbol{\theta} = (\phi_{1,0},...,\phi_{M,0},\boldsymbol{\psi},\alpha_1,...,\alpha_{M-1}),$ where $$\boldsymbol{\psi}$$ is the $$(qx1)$$ parameter vector containing contrained autoregressive parameters. As in the case of regular models, instead of the intercept parametrization that takes use of intercept terms $$\phi_{m,0}$$, one may use the mean parametrization with regimewise means $$\mu_m$$ instead (m=1,…,M).

### Examples of linear constraints

There are two main uses for linear constraints: restricting the autoregressive parameters to be the same for all regimes and constraining some parameters to zero. Of course many other arbitrary constraints may be useful, but we will show illustrative examples of these two.

#### Restricting AR-parameters to be the same for all regimes

To restrict the AR-parameters to be the same for all regimes, we want $$\boldsymbol{\phi_m}$$ to be the same for all m=1,…,M. The parameter vector $$\boldsymbol{\psi}$$ $$(qx1)$$ then corresponds to any $$\boldsymbol{\phi_m}=\boldsymbol{\phi}$$, and therefore $$q=pd^2$$. For the constraint matrix we choose $\boldsymbol{C} = [I_{pd^2}:\cdots:I_{pd^2}]' \enspace (Mpd^2xpd^2),$ that is, M pieces of $$(pd^2xpd^2)$$-diagonal matrices stacked on top of each other, because then $\boldsymbol{C}\boldsymbol{\psi}=(\boldsymbol{\psi},...,\boldsymbol{\psi})=(\boldsymbol{\phi},...,\boldsymbol{\phi}).$

#### Restricting AR-parameters to be the same for all regimes and constraining non-diagonal elements of coeffient matrices to be zero

The previous example shows how to restrict the AR-parameters to be the same for all regimes, but let’s say we also want to constrain the non-diagonal elements of coefficient matrices $$A_{m,i}$$ (m=1,…,M, i=1,…,p) to be zero. We have the constrained parameter $$\boldsymbol{\psi}$$ $$(qx1)$$ representing the unconstrained parameters $$(\boldsymbol{\phi_1},...,\boldsymbol{\phi_M})$$, where by assumption $$\boldsymbol{\phi_m}=\boldsymbol{\phi}=(vec(A_1),...,vec(A_p))$$ $$(pd^2x1)$$ and the elements of $$vec(A_i)$$ (i=1,…,p) corresponding to the diagonal are zero.

For illustrative purposes, let’s consider a GMVAR model with autoregressive degree p=2, number of mixture components M=2 and number of time series in the system d=2. Then we have $\boldsymbol{\phi}=(A_{1,(1,1)},0,0,A_{1,(2,2)},A_{2,(1,1)},0,0,A_{2,(2,2)}) \enspace (8x1) \enspace \text{and}$ $\boldsymbol{\psi}=(A_{1,(1,1)},A_{1,(2,2)},A_{2,(1,1)},A_{2,(2,2)}) \enspace (4x1).\quad\quad\quad\quad\quad\enspace$ By a direct calculation, we can see that choosing the constraint matrix $\boldsymbol{C}=\left[{\begin{array}{c} \boldsymbol{\tilde{c}} \\ \boldsymbol{\tilde{c}} \\ \end{array}}\right] \enspace (Mpd^2x4), \enspace \text{where}$

$\boldsymbol{\tilde{c}}=\left[{\begin{array}{cccc} 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array}}\right] \enspace (pd^2x4)$ satisfies $$\boldsymbol{C}\boldsymbol{\psi}=(\boldsymbol{\phi},...,\boldsymbol{\phi}).$$

## Functions in gmvarkit

### Estimating a GMVAR model

In the package gmvarkit one may use the function fitGMVAR to estimate a GMVAR model. The maximum likelihood estimation is performed in two phases: in the first phase fitGMVAR uses a genetic algorithm to find starting values for gradient based variable metric algorithm, which it then uses to finalize the estimation in the second phase. It’s important to keep in mind that it’s not guaranteed, that the numerical estimation algorithms will end up in the global maximum point rather than a local one. Because of multimodality and challenging surface of the log-likelihood function, it’s actually expected that most of the estimation rounds won’t find the global maximum point. For this reason one should always perform multiple estimation rounds. Parallel computing is used to obtain the results faster (number of cores used can be set with the argument ncores) and the number of estimation rounds can be controlled with the argument ncalls.

If the model estimates poorly, it is often because the number of mixture components M is chosen too large. One may also adjust the settings of the genetic algorithm employed, or set up an initial population with guesses for the estimates. This can by done by passing arguments in fitGMVAR to the (non-exported) function GAfit employing the genetic algorithm. To check the available settings, read the documentation ?GAfit. If the iteration limit is reached when estimating the model, the function iterate_more can be used to finish the estimation.

The parameters of the estimated model are printed in an illustrative and easy to read form. In order to easily compare approximate standard errors to certain estimates, one can print the approximate standard errors of the estimates in the same form with the function print_std_errors. Numerical approximation of the gradient and Hessian matrix of the log-likelihood at the estimates can be obtained conveniently with the functions get_gradient and get_hessian.

The estimated objects have their own print, plot and summary methods to name a few.

### Model diagnostics

The package gmvarkit considers model diagnostics based on multivariate extension of quantile residuals (see Kalliovirta and Saikkonen 2010), whose are under the correct model specification asymptotically multivariate standard normal distributed. The quantile residual tests introduced by Kalliovirta and Saikkonen (2010) can be performed with the function quantile_residual_tests by providing the estimated model (that is class gmvar object) as an argument.

For graphical diagnostics one may use the function diagostic_plot, which enables one to plot the quantile residual time series, auto- and cross-correlation functions of quantile residuals or their squares, or quantile residual kernel destity estimates and normal QQ-plots.

### Constructing GMVAR model without estimation

One may wish to construct an arbitrary GMVAR model without any estimation process, for example in order to simulate from a particular process of interest. An arbitrary model can be created with the function GMVAR. If one wants to add or update data to the model afterwards, it’s advisable to use the function add_data.

### Simulating from GMVAR process

The function simulateGMVAR is the one for the job. As the main argument it uses a gmvar-object (typically) generated by fitGMVAR or GMVAR.

### Forecasting GMVAR process

The package gmvarkit contains predict method predict.gmvar for forecasting GMVAR processes. For one step predictions using the exact formula for conditional mean is supported, but the forecasts further than that are based on independent simulations. The predictions are either sample means or medians and the confidence intervals are based on sample quantiles. The objects generated by predict.gmvar have their own plot method.

## References

• Kalliovirta L., Meitz M. and Saikkonen P. (2016) Gaussian mixture vector autoregression. Journal of Econometrics, 192, 485-498.
• Kalliovirta L. and Saikkonen P. (2010) Reliable Residuals for Multivariate Nonlinear Time Series Models. Unpublished Revision of HECER Discussion Paper No. 247.

1. For univariate analysis one may use the package uGMAR.