ar_model | Auto Regression Model implemention C | Testing library

Per Wikipedia:

The autoregressive model specifies that the output variable depends linearly on its own previous values and on a stochastic term (an imperfectly predictable term).

In your model example, you have one predictor - lagged value of y. In this simple case, the .predict() method multiplies each lagged value by the value of the estimated linear slope parameter for that predictor and adds the estimated value of the intercept of that line. So y_pred[10] will be equal to the product of the fitted slope parameter and y[9], with the value of the intercept estimate added.

Here is an example:

The code above returns a numpy array of [ 1 2 3 4 5 6 7 8 9 10 11 12], which I am interpreting as "The optimal lag p is 12" as per this StackOverflow question: stackoverflow.

Yes, that's right. The reason it returns an array instead of just 12 is that it can also search for models that do not include all of the lags, if you set glob=True. For example [ 1 2 3 12] might be a common result for a monthly model that has some annual seasonal pattern.

However, evaluating on some metrics (RMSE) I find that my AutoReg fitted models with maxlag=12 are performing worse than lower order models. By trial and error I found that the optimal lag is 8. So I am having difficulty interpreting the resulting numpy array, I have been reading the resources on statsmodels.com/ar_select_order and statsmodels.com/autoregressions but they have not made it clearer.

This function is returning the model that is judged optimal using information criteria. In particular, the default is BIC or Bayesian information criterion. If you use other criteria, such as minimizing the out-of-sample RSME, then it is definitely possible to find that a different model is judged to be optimal.

No setting start=len(train) is correct here. Note in this context that the indexing in Python starts at 0. Consequently, the last index available in your pandas series will be len(train)-1.

An easy way to validate this is by comparing the forecast from .predict() with the forecast computed by hand. Because I do not have access to your data, I will illustrate it with the sunspots example from the documentation. In there we estimate the following autoregressive model

• Why are the resulting coefficients different? Is it because of the estimation method? (Different settings of the method property of fit() don't give identical results)

AutoReg estimates parameters using OLS which is conditional (on the first observation) maximum likelihood. ARIMA implements full maximum likelihood and so uses the available information in the first observation when estimating parameters. In very large samples, the coefficients should be very close, and they are equal in their asymptotic limit. IN practice, they will always differ, although the difference should usually be minor.

• After getting the coefficients and backtesting the fitted results I match those of the AR(1) but not of ARIMA(1). Why?

The two models use different representations. AutoReg(1)'s model is Y(t) = a + b Y(t-1) + eps(t). ARIMA(1,0,0) is specified as (Y(t) - c) = b * (Y(t-1) - c) + eps(t). If |b|<1, then in the large sample limit c = a / (1-b), although in finite samples this identity will not hold exactly.

• What is ARIMA really doing in this simplest setting, isnt it supposed to be able to reproduce AR?

No. ARIMA uses the statsmodels Statespace framework which can estimate a wide range of models using Gaussian MLE.

ARIMA is essentially a special case of SARIMAX and this notebook provides a good introduction.

It is now a standalone function, statsmodels.tsa.ar_model.ar_select_order

https://www.statsmodels.org/devel/generated/statsmodels.tsa.ar_model.ar_select_order.html?highlight=ar_select

Your request might return 404. If that's the case, you can't decode the error pages. Therefore the following code might be a better choice:

Use that instead:

I will attempt an answer here. Unfortunately, writing formulas in SO is really cumbersome. So, firstly, I think you need to make a distinction between the "epsilon" of your DGP (data generation process), also called shocks or errors (which are unobservable) and the model residuals. Shocks are unobservable because you don't know the true values of the model parameters (in this case the AR and MA parameters) - you can only estimate them. So there is a difference (in a simple AR(1) model without a constant for ease of writing using a "hat" notation for estimates) between y_t - phi_1*y_{t-1} and y_t - \hat{phi_1}*y_{t-1} - the former being the "shock" and the latter being the "residual". Typically you would also have as notation \hat{y_t} = \hat{phi_1}*y_{t-1} and so residual defined as e_t = y_t - \hat{y_t}.

Now so, given that, I think what you would like to get is y_t - e_t - \hat{phi}*e_{t-1}, \hat{phi} being the estimated MA1 parameter. The following is an example where I have explicitly fitted an ARMA(2,1) without a constant to the LakeHuron time series. I have looked at the model residuals using residuals (resid in df) and also calculated by hand (resid2 in df) - they differ a bit due to approximating initial conditions that the residuals method makes which I didn't really dig into. You will see that the values get closer and closer as t increases and the influence of the initial conditions diminishes (this is due to the fact that we have a stationary ARMA model). In the same way I have calculated the "AR_fit" as I call it once as ar_1 * y_{t-1} + ar_2 * y_{t-2} (ar_fit1 in df) and a second time as y_t - e_t - ma_1 * e_{t-1} (ar_fit2 in df). Again, you will see that the values converge as t increases (due to the same initial conditions used to calculate the residuals using the residuals function).

In the list of object files that are linked there is one file missing: rcpp_functions.o. You could add this to OBJECTS in the Makevars file: