estimation of ARMA(1,1) with MA parameter greater than 1

I am working with the following simple ARMA(1,1) model: $$z_{t+1} = \phi z_{t} + \theta\varepsilon_{t} + \varepsilon_{t+1}$$ In my case $$\theta$$ depends on some other parameters, and, therefore, I know that $$\theta > 1$$.

Let us generate 1000 ARMA(1,1) samples with $$\phi = 0.95$$, $$\theta = 0.5$$, $$\sigma = 0.08$$ and estimate the parameters.

import numpy as np
import statsmodels.api as sm
from statsmodels.tsa.arima_process import ArmaProcess

process = ArmaProcess(np.r_[1, -0.95], np.r_[1, 0.5])
y = process.generate_sample(1000, scale=0.08)
model = sm.tsa.ARMA(y, (1, 1)).fit(trend='nc', disp=0)


The result is [0.96069232 0.51912881 0.080], which is quite close to the true values.

Next, lets us try parameters $$\phi = 0.95$$, $$\theta = 2$$, $$\sigma = 0.08$$, i.e. change the following row in the code above:

process = ArmaProcess(np.r_[1, -0.95], np.r_[1, 2])


The result is [0.95668055 0.49700497 0.156] and there is no errors and warnings!

By default, in order to have distinguishable auto-covariance function we assume $$|\theta| < 1$$. Therefore, in the result the estimator is close to $$\frac{1}{\theta}$$. Though, the standard deviation of the variance is far from the true value.

Is there a way to estimate the case when $$\theta > 1$$? From what I can see, all packages use the fact that $$|\theta| < 1$$.

I suspect that the reason is that many ARMA fitting algorithms require that the ARMA system be stably invertible. In your example, the MA process is not minimum phase as the zero will be outside the unit circle. That makes the inverted system unstable (the pole will be outside the unit circle).

I don't know the details of the sm.tsa.ARMA(y, (1, 1)).fit algorithm, but I suspect that it will also be so restricted.

See, for example, this lecture by Florian Pelgrin, which contains the slide below.

In particular, note the second condition.

This means that I suspect it will be harder to find an algorithm that does the estimation for the case you're interested in.

• I am very impressed that the model itself is not very complicated, but the estimator is very tricky... – ABK Nov 6 '19 at 16:09
• @ABK Yes, estimation is remarkably difficult for such an easy-to-pose question. – Peter K. Nov 6 '19 at 16:15
• @ABK: Any code that maximizes a non-linear function with linear constraints is not going to be trivial. Your case is probably no more complicated than the case Python covers. So, if you look at that code, chances are that you can figure out how to handle the opposite constraint. These algorithms often use some kind of variable metric approach ( e.g; BFGS ) where they step some amount and then use the hessian, to choose direction. Each time a new point is reached, then a check is done to see how close the gradient is to zero. So, definitely complex but also pretty standard in optimization. – mark leeds Nov 6 '19 at 22:58
• Another complication is that the characteristic root of the AR polynomial also have to satisfy a constraint. The AR constraint ends up being straightforward in your case but not for an AR(p). – mark leeds Nov 6 '19 at 23:00
• thank you! Nevertheless, it is so strange that almost no one used this simple model... – ABK Nov 7 '19 at 14:03