I'm taking a DSP course and we're being introduced to the whole probability/stats side of DSP and I'm just confused on things like the ARMA models. These things don't seem intuitive to me at all.

First of all, what exactly is the point of the ARMA model? Just to predict what future values of x[n] (the input) will be?

Secondly, I saw this as an inuitive explanation for the ARMA model, enter image description here

My question is, why is v[n] in there (white gaussian noise)? I understand why v[n-1] and previous values of WGN are there but how can you get the current value of the noise?

Everything seems really weird. I don't really see the point of how any of this or all that least-mean square algorithim stuff is useful. Then theres the Yule-Walker equations which are used to estimate the parameters for the ARMA model. I just don't see how all this stuff fits in with each other.

  • $\begingroup$ If you were to assign an ARMA model to a white noise sequence, what do you think it would be? $\endgroup$ – user28715 Nov 13 '18 at 20:55

It's not very possible to give a summary of the whole statistical signal processing in a single answer here, as those topics you mention span a great deal of it; ARMA modeling, linear prediction, Yule-Walker equations, Least Square and Mean Square design criterions etc.

Indeed, it's true that all these different named concepts look quite similar and revolve around very same looking equations from there to here. And that possibly adds a further complexity into one's already existing confusion, as it did in most of ours'... But that can be away most probably only after you complete the course. Anyway.

From a practical point of view, a discrete-time ARMA(p,q) random process $x[n]$ is nothing but the output of an LTI filter with coefficients $a_k$ and $b_k$ when the input is a white noise of unit variance, typically denoted as $v[n]$.

The LTI filter has the following system function:

$$ H(z) = \frac{B(z)}{A(z)} = \frac{ \sum_{k=0}^{k=q} b_k z^{-k} }{\sum_{k=0}^{k=p} a_k z^{-k} }= \frac{ b_0 + b_1 z^{-1} + ... + b_q z^{-q} }{ 1 + a_1 z^{-1} + ... + a_p z^{-p}} $$

The process is produced as the following: $$ v[n] \longrightarrow \boxed{ H(z) } \longrightarrow x[n] $$

Since the input has $\sigma_v^2 = 1$, $r_{vv}[k] = \delta[k]$, and $ S_{xx}(e^{j \omega} ) = 1 $ then it can be shown that (from LTI processing of random processes) the output metrics will be $$r_{xx}[k] = h[k] \star h^*[-k] $$ and $$ S_{xx}(e^{j \omega} ) = |H(e^{j \omega} )|^2 $$ where $h[n]$ is the impulse response of the LTI filter.

Two nice things about this ARMA modeling is that, first it's easy to calculate the statistical metrics of it, and second when you determine those coefficients $a_k$ and $b_k$ associated with an arbitrary random process $x[n]$, then the entire random process is replaced with just those $p+q+1$ variables $a_k$ and $b_k$ and some modeling error sequence. Such a modeling might yield great simplifications in various processing stages.

  • $\begingroup$ Ah I see. So, the moving-average model on it's own is trying to estimate the actual noise of my system with a real input by pushing WGN through some filter coefficients. And once I am able to find this estimate of the noise, I can subtract it from my real input and hence get rid of noise?? If that's the case then let's say for the folloiwng moving average model , x[n] = v[n] + b1v[n-1] ..... in which x[n] is output and v[n] is WGN input. What's the point in getting previous values of v[n] if it's all WGN? Is it to try and adjust the b parameters are make x[n] more like my real noise? $\endgroup$ – AlfroJang80 Nov 14 '18 at 1:57
  • $\begingroup$ would that be correct? $\endgroup$ – AlfroJang80 Nov 16 '18 at 17:14
  • $\begingroup$ I'm still a bit unclear on what exactly is the point of giving an input as the white noise? Where does our actual signal come in $\endgroup$ – AlfroJang80 Nov 16 '18 at 17:23
  • $\begingroup$ it's the definition of an ARMA process. There is an LTI filter with impulse response $h[n]$ , or equivalently an LCCDE of type $$\sum_{k=0}^{p} a_k x[n-k] = \sum_{k=0}^{q} b_k v[n-k] $$ where the input $v[n]$ is a white-noise of unit variance and the output $x[n]$ is the ARMA type random process. It's just a model of a random process. There are many other models. But this one is quite useful in certain conexts. What makes $x[n]$ a random process is the input being a random signal and specifically a white noise. You can also input other random signals but that won't be called ARMA. $\endgroup$ – Fat32 Nov 16 '18 at 20:44
  • $\begingroup$ AlfroJang80: This is in response to your comment at 1:57. In your moving average model, you're not trying to make the x[n] be close to white noise. You have all of the x[n] because they are the series and you try to find the values of the $b_i$ so that your estimates, $\hat{x}[n]$, in your estimated model are as close to the actual x[n] as possible in terms of squared loss. $\endgroup$ – mark leeds Dec 14 '18 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.