I am working with the following simple ARMA(1,1) model: $$ z_{t+1} = \phi z_{t} + \theta\varepsilon_{t} + \varepsilon_{t+1} $$ In my case $\theta$ depends on some other parameters, and, therefore, I know that $\theta > 1$.

Let us generate 1000 ARMA(1,1) samples with $\phi = 0.95$, $\theta = 0.5$, $\sigma = 0.08$ and estimate the parameters.

import numpy as np
import statsmodels.api as sm
from statsmodels.tsa.arima_process import ArmaProcess

process = ArmaProcess(np.r_[1, -0.95], np.r_[1, 0.5])
y = process.generate_sample(1000, scale=0.08)
model = sm.tsa.ARMA(y, (1, 1)).fit(trend='nc', disp=0)

The result is [0.96069232 0.51912881 0.080], which is quite close to the true values.

Next, lets us try parameters $\phi = 0.95$, $\theta = 2$, $\sigma = 0.08$, i.e. change the following row in the code above:

process = ArmaProcess(np.r_[1, -0.95], np.r_[1, 2])

The result is [0.95668055 0.49700497 0.156] and there is no errors and warnings!

By default, in order to have distinguishable auto-covariance function we assume $|\theta| < 1$. Therefore, in the result the estimator is close to $\frac{1}{\theta}$. Though, the standard deviation of the variance is far from the true value.

Is there a way to estimate the case when $\theta > 1$? From what I can see, all packages use the fact that $|\theta| < 1$.


1 Answer 1


I suspect that the reason is that many ARMA fitting algorithms require that the ARMA system be stably invertible. In your example, the MA process is not minimum phase as the zero will be outside the unit circle. That makes the inverted system unstable (the pole will be outside the unit circle).

I don't know the details of the sm.tsa.ARMA(y, (1, 1)).fit algorithm, but I suspect that it will also be so restricted.

See, for example, this lecture by Florian Pelgrin, which contains the slide below.

Slide 6 from Florian Pelgrin's presentation.

In particular, note the second condition.

This means that I suspect it will be harder to find an algorithm that does the estimation for the case you're interested in.

  • 1
    $\begingroup$ I am very impressed that the model itself is not very complicated, but the estimator is very tricky... $\endgroup$
    – ABK
    Nov 6, 2019 at 16:09
  • $\begingroup$ @ABK Yes, estimation is remarkably difficult for such an easy-to-pose question. $\endgroup$
    – Peter K.
    Nov 6, 2019 at 16:15
  • $\begingroup$ @ABK: Any code that maximizes a non-linear function with linear constraints is not going to be trivial. Your case is probably no more complicated than the case Python covers. So, if you look at that code, chances are that you can figure out how to handle the opposite constraint. These algorithms often use some kind of variable metric approach ( e.g; BFGS ) where they step some amount and then use the hessian, to choose direction. Each time a new point is reached, then a check is done to see how close the gradient is to zero. So, definitely complex but also pretty standard in optimization. $\endgroup$
    – mark leeds
    Nov 6, 2019 at 22:58
  • $\begingroup$ Another complication is that the characteristic root of the AR polynomial also have to satisfy a constraint. The AR constraint ends up being straightforward in your case but not for an AR(p). $\endgroup$
    – mark leeds
    Nov 6, 2019 at 23:00
  • 1
    $\begingroup$ thank you! Nevertheless, it is so strange that almost no one used this simple model... $\endgroup$
    – ABK
    Nov 7, 2019 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.