Reading the ARMA model for the first time, and I'm confused.

Let's say I have a time series

x = [1, 2.1, 2.9, 3, 4.1]

According to the ARMA model, $X_t$ is a linear combination of previous values and errors, something like

$$\sum_i \phi_i X_{t-i} + \sum_i \theta_i \epsilon_{t-i}$$

But, what are $X_i$ and $\epsilon_i$??

  • is $X_i$ the actual value of the series at $t=i$? Eg, in my example $X_2 = 2.1 ?? (with 1-based indexing)
  • if so, what are the error terms?

The same question for the simple moving-average model, where all the $X_i$ values are remain unused.


In the context of discrete-time statistical signal processing, an ARMA-(p,q) random process is defined as (assuming zero initial conditions)

$$ \sum_{k=0}^p a_k ~ x[n-k] = \sum_{k=0}^q b_k ~ v[n-k] $$

or equivalently

$$ x[n] = - \sum_{i=1}^p a_i ~ x[n-i] + \sum_{i=0}^q b_i ~ v[n-i] $$

where $v[n]$ is a white-noise (WSS) random process with variance $\sigma_v^2$ and $x[n]$ represents the resulting ARMA process.

In this context your coefficients are related as $$\phi_i = - a_i $$ and $$\theta_i = b_i$$ The process reggresses over its past values $x[n-i]$ ($X_{t-i}$ acc) and is also a moving average of the input noise $v[n-i]$ ($\epsilon_{t-i}$ acc).

  • $\begingroup$ That was a nice way of representing the ARMA in a DSP framework. One thing I don't get. Is it possible to define the impulse response of an ARMA since the model has both noise terms and it's own lagged terms ? It doesn't seem possible to me. Any comments or references are appreciated because the time-series literature, AFAIK, doesn't discuss the impulse response much. Thanks. $\endgroup$ – mark leeds Oct 13 '18 at 1:03
  • $\begingroup$ @Fat32: still, my question is, where do the values of the noise come from? Do I randomly generate them during regression, instead of getting them from external source? And, if so, what is the point? $\endgroup$ – blue_note Oct 13 '18 at 10:44
  • $\begingroup$ Now, the error you mention is the white-noise sequence here. First the white-noise of sufficeint length is generated then it's filtered by $x = filter(b,a,w)$ to get the ARMA-process. In this perspective, ARMA is used to gererate x from w. But in another perspective it can be used to model x[n] up to error terms w[n]. So it depends on how you view the process. yes @markleeds once the ARMA coefficients $a$ and $b$ are defined, the following Matlab/Octave command h = impz(b,a,M) produces the impulse response truncated to the length M. $\endgroup$ – Fat32 Oct 13 '18 at 14:26
  • $\begingroup$ @Fat32: Thanks. That's useful if I ever use Matlab but I'm interested in the theoretical DSP derivation. I'll google for "impulse response and ARIMA models" and see if anything useful comes up. If so, I'll post any good links. $\endgroup$ – mark leeds Oct 14 '18 at 4:42
  • 1
    $\begingroup$ @Fat32: it just arrived and I only glanced at it for a half hour or so but Monson Hayes' text is the best book I've seen as far as providing a clear "dsp in time series" viewpoint. It's quite enlightening to see that viewpoint presented in a clear way. Thanks. P.S: To any new DSPers: I don't have Kay which is supposed to be good also. See thread above for links. $\endgroup$ – mark leeds Oct 26 '18 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.