0
$\begingroup$

I have similar questions as the one asked in these posts: https://stackoverflow.com/questions/47083890/fir-filter-length-is-the-intercept-included-as-a-coefficient-matlab/47085339?noredirect=1#comment81124362_47085339 and Terminologies - lags, order in time series model

But the answer is quite different from the one in books and other online resources such as this tutorial: https://onlinecourses.science.psu.edu/stat510/node/47

If the time series model is of order 2, then according to the answers to the earlier question, there should be 3 coefficients. But, the tutorial link says differently. In the link an AR(1) model has 1 coefficient.

In a research article titled, "X. Xu and J. Guo, "A Fast System Identification Method Based on Minimum Phase Space Volume," 2012 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, Sanya, 2012, pp. 523-526." the Authors have doen system identification of AR(2) model. There they have used 2 coefficients to express the model.

Based on my understanding, the number of coefficients that a system has is known as the length, $L$ and the order is $q=L-1$.

I want help to confirm what is the correct representation and terminology. In general for AR and MA models,

Confusion 1) For an AR(1) system of order $q=1$, should there be 2 coefficients or 1 coefficient? Same thing for MA(1)

According to the online course link for AR(1), there is only 1 coefficient.

Confusion 2) What is correct? Considering an AR(2) model having coefficients [a1,a2,a3] then can I express the model as: x[t] = a1*x[t] + a2*x[t-1] + a3*x[t-2]+ e[t] and for MA(2) of order 2 as x[t] = a1*e[t] + a2*e[t-1] + a3*e[t-2]

where e[t] is the excitation input driving signal.

What is the correct method? Please help, I am extremely confused.

$\endgroup$
1
$\begingroup$

Hi: As far as what you wrote in confusion 2):

The AR expression should not have an $x[t]$ on both sides. For example, below is an AR(2):

$x[t] = a1*x[t-1] + a2*x[t-2]+ e[t]$

Similarly, the MA expression should always have a coefficient of 1 on the $e[t]$ term. So, for example, an MA(2) would be:

$x[t] = e[t] + a1*e[t-1] + a2*e[t-2]$

As Stanley said, it is only a matter of convention but the order of an AR(p) is generally thought of as $p$ so there are $p$ AR coefficients if the order is p. Similarly, for an MA(q) there are q MA coefficients so the order is considered to be q ( the $e[t]$ is not included in the order for an MA because there is no coefficient associated with $e[t]$).

Note that, for both MA and AR models, the number of parameters to be estimated is not always equal to the order of the model because there may be non zero mean terms or trend terms. For example, below is AR(2) with unknown mean $\mu$:

$x[t] = \mu + a1*x[t-1] + a2*x[t-2]+ e[t]$.

Note that there are also ARMA models where the number of coefficients is then p+q. For example, an ARMA(2,1) is:

$x[t] = a1*x[t-1] + a2*x[t-2]+ b1*e_[t-1] + e[t]$.

But usually the term "order" is not used for ARMA models.

I hope this makes it clear.

$\endgroup$
  • $\begingroup$ Thank you for your answer, but I have some questions could you please clarify?(1)I think there might be a typo in the representation for MA(2). The formula for MA model is $x[t]=\sum_{q=0}^{L-1} a_i e[t-i]$ and as per the the link number 2 in my question from dsp site, first coefficient $a1$ should be multiplied to $e[t]$. So, I think the MA(2) should be $x[t] = a1*e[t] + a2*e[t-1]+a3*e[t-2]$. This equation has 3 coefficients but it is not always true that the coefficient for $e[t]$ will always be 1 as per the answer in the stackexchange site). $\endgroup$ – SKM Dec 5 '17 at 22:45
  • $\begingroup$ However, you say that the order $q$ means there are $q$ number of coefficients. Due to these different representations, it is still not clear where the order equals the number of coefficients or not. (2) Does the order have a conection to the lags or the integer number denoted as delays or lags? In other words, for the MA and AR models, we write $t-1,t-2$ and multiply them with coefficients. These integere number 1,2 etc are the lags? If so, what is the physical meaning of lags? Thank you for your time and effort. $\endgroup$ – SKM Dec 5 '17 at 22:54
  • $\begingroup$ @SKM: Hi This answers your first comment I hope. It's not possible to have a coefficient on the $e[t]$ term because it occurs at the same time as the response so there's no way to estimate the coefficient. You can think of the $e[t]$ term as the pure noise part of the signal so don't think of it as an MA term because it isn't one. You will never see a coefficient on the $e[t]$ in an MA model no matter where you look. I can guarantee that. I don't mean to sound rude but whatever the stackexchange link says is incorrect. $\endgroup$ – mark leeds Dec 6 '17 at 6:36
  • $\begingroup$ @SKM: it's not a bad idea to think of the AR or MA terms as being on lagged values of the response or error terms. But generalizations such as that don't always hold. For example: one could have: $y[t ]= y[t-1] + b1 * *(y_[t-2] - y[t-3]) + e[t]. Here, the coefficient on the first lagged term is 1.0 and the first difference is AR(1). I don't think it's worth trying to generalize . The whole AR and MA framework developed by Box and Jenkins is about building models by using previous values of the response or error terms. Maybe take a look at the box and jenkins text if I'm not so clear ? $\endgroup$ – mark leeds Dec 6 '17 at 6:46
0
$\begingroup$

Confusion 1:

What is the power spectra?

$$ \mid H(\omega)\mid^2=\frac{\sigma^2}{\mid 1+a e^{-\jmath \omega}\mid^2} $$ we have $a$ for the feedback parameter, but also $\sigma$ which is associated with $e(t)$ which is supposed to be white noise. So if I'm estimating an AR process, I'll probably need to estimate $\sigma$, as well as $a$. If I'm modeling a process, $\sigma$ lets me scale the power spectrum. Some authors will specify the $\sigma$ for $e(t)$ explicitly.

Confusion 2:

Your $x(t)= a_1 x(t) + a_2 x(t-1) + a_3 x(t-3) + e(t)$ can be rewritten as $(1-a_1)x(t)= a_2 x(t-1) + a_3 x(t-3) + e(t)$ and if I divide both sides by $1-a_1$ I'll get

$$ x(t)-b_1x(t-1) - b_2 x(t-2) - b(3) x(t-3)= C e(t) $$ which is the form more typically used.

The term associated with $x(t)$ is most usually $1$ so the denominator of the power spectrum is usually of the form $1 + \sum_{i=1}^N a_i e^{\jmath \omega i}$

In the MA model, you want to scale each of the $e(t) \; e(t-1) \dots $ terms.

As a general comment, these are conventions, and conventions are adopted by communities, so there isn't really a correct or incorrect convention. You need to just carefully look at the paper and understand what it is saying knowing that the actual words might be a little different than what you think they are. In comment 1, your confusion seems to be related to the variance of $e(t)$ as opposed to the lag multiplier terms.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.