There is a random process, $X:\mathbb{Z}\rightarrow\mathbb{R}$ in discrete-time domain, which is wide-sense stationary with zero mean and autocorrelation function $R_X(\tau)$.

a) What range of values can $R_X(1)$ take? For this, using the relationship $|R_X(\tau)|\leq R_X(0)$, I derived $-1\leq a \leq 1$.

b) If $R_X(0)=1$ and $R_X(1)=a$, show that $2a^2-1\leq R_x(2) \leq 1$.

I can understand its upper bound, but I do not know how I can derive the lower bound $2a^2-1$.

Could anyone help with this?

  • $\begingroup$ Hint: what is the possible range of values for a? $\endgroup$ Jan 16, 2022 at 20:47
  • $\begingroup$ @GrapefruitIsAwesome That is derived from (a) that $-1\leq a\leq 1$. $\endgroup$
    – Junho
    Jan 16, 2022 at 21:01
  • $\begingroup$ Yes, now substitute those bounds into the expression for Rx(2) $\endgroup$ Jan 16, 2022 at 21:04
  • $\begingroup$ @GrapefruitIsAwesome I don't get it.. If $a=1$, then $R_X(2)$ is always 1. Why should it be true? $\endgroup$
    – Junho
    Jan 16, 2022 at 21:06
  • $\begingroup$ Hi: Note that you are defining the statistical definition of autocorrelation rather than the DSP definition.( which is the auto-covariance in statistics ). If $a = 1$ then the autocorrelation at lag 1, $R_{x}(1)$ , $ = 1.0$ which implies that the process is some constant value so the autocorrelations at all lags are 1.0. But, as far as proving that bound you wrote, I tried and was unsucccessful. $\endgroup$
    – mark leeds
    Jan 16, 2022 at 21:44

2 Answers 2


Some definitions: $$R_{X}(0) = 1$$ $$R_{X}(1) = a$$ $$R_{X}(2) = b$$

I will use the the condition that the autocorrelation be positive semi-definite, and the above definitions for all parts of the problem.

The derivation below is an application of Sylvester's criterion.

Lag 0 $$\begin{vmatrix} 1 \end{vmatrix} \geq 0\hspace{1cm} \checkmark$$

Lag 1 $$\begin{vmatrix} 1 & a\\ a & 1 \end{vmatrix} \geq 0$$ $$1 - a^{2} \geq 0$$ $$a^{2} \leq 1$$ $$-1 \leq a \leq 1\hspace{1cm} \checkmark$$

Lag 2 $$\begin{vmatrix} 1 & a & b \\ a & 1 & a \\ b & a & 1 \end{vmatrix} \geq 0$$ $$1 + a^{2}b + a^{2}b - b^{2} - a^{2} - a^{2} \geq 0$$ $$-b^{2} + 2a^{2}b - 2a^{2} + 1 \geq 0$$ $$b^{2} - 2a^{2}b + \left(2a^{2} - 1\right) \leq 0$$

Solving for the equality first:

$$b = \frac{2a^{2} \pm \sqrt{4a^{4} - 4\left(2a^{2} - 1\right)}}{2} $$ $$b = a^{2} \pm \sqrt{\left(a^{2}-1\right)^{2}}$$ $$b = a^{2} \pm \left(a^{2}-1\right)$$ $$b \in \{2a^{2} - 1, 1\}$$

The solution is valid between these bounds and invalid outside so:

$$2a^{2} - 1 \leq b \leq 1$$

Using our original definition:

$$2a^{2} - 1 \leq R_{X}(2) \leq 1\hspace{1cm} \checkmark$$

  • $\begingroup$ @GrapefruitsAwesome: In addition to what Matt said, you also didn't justify the first expression in your series of expressions , $| 2 a^2 -1 | \le 1$. $\endgroup$
    – mark leeds
    Jan 17, 2022 at 15:43
  • $\begingroup$ @MattL. I've reworked the derivation to derive all of the portions of the solution using positive semi-definiteness $\endgroup$ Jan 18, 2022 at 0:56
  • $\begingroup$ @GrapefruitIsAwesome: Ok, great. I think I've found out why our answers seem to disagree. In fact, they don't. It's just about necessity and sufficiency of the given condition. The given limits on $R_X[2]$ are not sufficient. Assuming that $R_X[k]=0$ for $|k|>2$ (as I did in my example) shows that the resulting sequence is not a valid autocorrelation sequence. $\endgroup$
    – Matt L.
    Jan 18, 2022 at 12:00
  • $\begingroup$ See my updated answer. $\endgroup$
    – Matt L.
    Jan 18, 2022 at 12:07
  • $\begingroup$ GrapefruitIsAwesome: I have to go through it more carefully later but thanks. That's a great answer and I never knew that about the autocorrelations. Are there similar relations at every lag ? Also, the 1's on the diagonal that you use tells me that you're using statistical autocorrelations. There are a lot of threads on this list that get long because DSP defines autocorrelation as the thing that statisticians call autocovariance. This can cause confusion. $\endgroup$
    – mark leeds
    Jan 18, 2022 at 15:58

Let me give you a counterexample showing that the given limits for $R_X[2]$ in terms of $R_X[0]$ and $R_X[1]$ are not sufficient for guaranteeing the non-negativity of the corresponding power spectrum.

Let's fix $R_X[2]$ at its given lower limit:


Furthermore, let's assume that $R_X[k]=0$ for $|k|>2$. The DC value of the corresponding power spectrum $S_X(\omega)$ is just the sum over all values $R_X[k]$, and is given by


It's straightforward to show that for


the value of $S_X(0)$ becomes negative, which shouldn't be the case for a power spectrum. Hence, it appears that the given lower bound for $R_X[2]$ is not correct.

EDIT: As shown in GrapefruitIsAwesome's answer, it appears that the given limits on $R_X[2]$ have been derived from the requirement that the autocorrelation matrix be positive semi-definite. However, for the sequence $R_X[k]$ to be a valid autocorrelation sequence, it is necessary that all Toeplitz matrices with first row $R_X[0],R_X[1],\ldots,R_X[n]$ are positive semi-definite for $n=1,2,\ldots,\infty$.

In the example above I've assumed $R_X[k]=0$ for $|k|>2$, in which case the given limits on $R_X[2]$ are not sufficient, but only necessary. The given limits just guarantee that there exist valid autocorrelation sequences with the first three elements as given. However, not all sequences with those first three elements are valid autocorrelation sequences.

  • $\begingroup$ Oh, in this case, $R_X[k]$ for $|k|>2$ is not defined, so I do not know whether we can use the power spectral density properly. Thank you for the answer, by the way. $\endgroup$
    – Junho
    Jan 16, 2022 at 22:00
  • $\begingroup$ Matt: Is that true that $S_{x}(0)$ is the sum over the autocorrelations even when the autocorrelations are the statistical ones ? I know its true for statistical autocovariances which are called autocorrelations in the DSP world. But ignore this if it makes no sense because I find the DSP definitions confusing and I could be confused. I still can't prove his expression using statistical autocorrelations so maybe it is flawed ? $\endgroup$
    – mark leeds
    Jan 17, 2022 at 15:48
  • $\begingroup$ @markleeds: How do you see which definition of autocorrelation the OP uses? $\endgroup$
    – Matt L.
    Jan 17, 2022 at 17:57
  • $\begingroup$ The statistical autocorrelation at lag zero is always 1 so that's why I figured that he's using statistical autocorrelations. As far as I can tell from being on this list ( but don't claim expertise in DSP auto-corr ) the dsp autocorrelation is the statistical autocovariance and I don't know what DSP uses for the statistical autocorrelation. This link clarifies the two definitions. The "definition for wide sense stationary stochastic process" is the DSP one and the normalized one is the statistical one. en.wikipedia.org/wiki/Autocorrelation. In statistics, we don't call it normalized. $\endgroup$
    – mark leeds
    Jan 18, 2022 at 15:52
  • $\begingroup$ Matt: Just to close the discussion on this difference. The reason I think that the DSP use of the term autocorrelation is poor is because the DSP autocorrelation at lag zero is the variance ( by definition ) so, to be in sync with this and the rest of the terms ( which are covariances by definition ), the term auto-covariance, used in statistics, makes more sense because it conveys the meaning more clearly. $\endgroup$
    – mark leeds
    Jan 18, 2022 at 23:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.