Noise added to a Random Process

if we have a discrete random process $$$$x\left(n\right)\:=\:0.2x\left(n-1\right)+w\left(n\right)+w\left(n-1\right)$$$$ where $$w\left(n\right)$$ is a noise with a mean $$m_w=0.5$$ and variance $$\sigma ^2_w\:=\:1$$.

I am trying to find the value of $$E\left\{x^2\left(n\right)\right\}$$.

Here's my work below:

$$$$E\left\{x^2\left(n\right)\right\} = E\bigg\{\big[0.2x\left(n-1\right)+w\left(n\right)+w\left(n-1\right)\big]^2\bigg\}$$$$

for $$E\big\{w\left(n-i\right)^2\big\}=\sigma _w^2+m_w^2=1+0.5\:=\:1.5.$$

After that, I get the following \begin{align} E\left\{x^2\left(n\right)\right\} =&\phantom{+} 0.04 E\left\{x^2\left(n-1\right)\right\}\\ &+0.4E\left\{w\left(n\right)x\left(n-1\right)\right\}\\ &+0.4E\left\{w\left(n-1\right)x\left(n-1\right)\right\}\\ &+E\left\{w^2\left(n\right)\right\}\\ &+2E\left\{w\left(n\right)w\left(n-1\right)\right\}\\ &+E\left\{w^2\left(n-1\right)\right\} \end{align}

• Comments are not for extended discussion; this conversation has been moved to chat.
– Peter K.
Nov 9, 2020 at 13:39

Hi: I don't know whether white noise can have a non-zero mean ( Dilipe can tell us that ) but, if it can't, then just call it whatever you call white noise with a non-zero mean.

But, assuming it's zero, this is how you can do it. ( If it's not zero, then the solution is similar but the last step would be a little different. Use $$\operatorname{Var}\left\{X\right\} = E\left\{X^2\right\} - E\left\{X\right\}^2$$ Note also I wrote it with "t's" and "epsilon's" just because I'm more used to those but you can think of $$t$$ as $$n$$ and $$\epsilon$$ as $$w$$.

\begin{align} x_t &= 0.2 x_{t-1} + \epsilon_{t} + \epsilon_{t-1}\\ \implies x_{t}(1 - 0.2L) &= \epsilon_{t} + \epsilon_{t-1} \tag{\scriptsize{\text{using lag operator}}}\\ \implies x_{t} &= \frac{\epsilon_{t}}{1 - 0.2L} + \frac{\epsilon_{t-1}}{1 - 0.2L}\\ \implies x_{t} &= \sum_{i = 0}^{\infty} 0.2^{i} \epsilon_{t-i} + \sum_{i = 0}^{\infty} 0.2^{i} \epsilon_{t-1-i} \end{align}

So, since the mean is zero, the $$E\left\{x^2_{t}\right\}$$ is the variance of the RHS.

So, it is \begin{align} \sum_{i = 0}^{\infty} \left(0.2^{i}\right)^2 \cdot \sigma^2_\epsilon + \sum_{i = 0}^{\infty} \left(0.2^{i}\right)^2 \cdot \sigma^2_\epsilon&= \sum_{i = 0}^{\infty} 0.04^{i} \cdot \sigma^2_\epsilon + \sum_{i = 0}^{\infty} 0.04^{i} \cdot \sigma^2_\epsilon\\ &= \frac{2 \cdot \sigma^2_{\epsilon}}{(1- 0.04)} \end{align}

• "white" means: Spectrum is white. Something that has a mean has a peak in the spectrum at f=0. Or, more strictly speaking, something is white exactly if it's WSS and its autocorrelation function is zero everywhere but for zero lag, but if your process has a mean, then the autocorrelation can't be zero – you could always "take apart" your process into a constant + a zero-mean process, and then you'd end up with constant² + 0 as autocorrelation – and that's not zero. Nov 8, 2020 at 17:58
• For a discussion on the definition of white noise take a look at this question and its answers. Nov 8, 2020 at 19:01
• @MattL. right! I almost forgot about that! Nov 8, 2020 at 19:49
• Matt and Marcus: Thanks for white-noise references and explanations. Nov 8, 2020 at 22:14
• I'll read links but , in statistics, you can have have a zero mean process + a constant and there's still no autocrorrelation but that's get into the weirdness of the definitions of autocorrelation in statistics versus DSP. The assumption in my derivation is that $\epsilon_t$ is an iid zero mean process with $E(\epsilon_t \epsilon_s) = 0 ~\forall t \ne s$. This way, I stay away from the whole noise definition and autcorrelation definitions of DSP. Nov 8, 2020 at 22:29

Let's define a helper: $$y(n) = w(n) + w(n-1)$$. Then, your $$x$$ becomes

\begin{align} x(n) &= \frac15 x(n-1) + y(n)\\ &= \frac15 \left(\frac 15x(n-2)+y(n-1)\right) + y(n)\\ &=\frac 1{25}x(n-2)+\frac15 y(n-1)+ y(n)\\ &=\frac1{5^3}x(n-3)+\frac 1{25}y(n-2)+\frac15 y(n-1)+ y(n)\\ &=\sum_{p=0}^\infty 5^{-p}y(n-p)\\ &=\sum_{p=0}^\infty\left( 5^{-p}w(n-p)+5^{-p}w(n-p-1)\right)\\ &=\sum_{p=0}^\infty 5^{-p}w(n-p)+\sum_{q=1}^\infty5^{-q+1}w(n-q)\\ &=5^{-0}w(n)+\sum_{p=1}^\infty 5^{-p}w(n-p)+\sum_{q=1}^\infty5^{-q+1}w(n-q)\\ &=w(n)+\sum_{p=1}^\infty \left(5^{-p}w(n-p)+5^{-p+1}w(n-p)\right)\\ &=w(n)+\sum_{p=1}^\infty w(n-p)\left(5^{-p}+5^{-p+1}\right)\\ &=w(n)+6\sum_{p=1}^\infty w(n-p)5^{-p}\\[2em] % % E\left[x^2(n)\right] &= E\Big[\Big(w(n) +\underbrace{\sum_{p=1}^\infty w(n-p)6\cdot5^{-p}}_{s(n)}\Big)^2\Big]\\ &= E[w^2(n)] + E[2w(n)s(n)] + E[s^2(n)]\\ &= \frac32 + 2E[w(n)s(n)] + E[s^2(n)]\\ &= \frac32 + 2E\left[\sum_{p=1}^\infty w(n)w(n-p)6\cdot5^{-p}\right] + E[s^2(n)]\\[2.5em] % % E[s^2(n)] &= E[s(n)\cdot s(n)]\\ &\text{Cauchy product formula:}\\ &= E\left[\sum_{k=0}^\infty\sum_{l=1}^{k-1} w(n-l)\left(5^{-l}+5^{-l+1}\right) w(n-k+l)\left(5^{-k+l}+5^{-k+l+1}\right)\right]\\ &= E\left[\sum_{k=0}^\infty \sum_{l=1}^{k-1}\left(5^{-l}+5^{-l+1}\right)\left(5^{-k+l}+5^{-k+l+1}\right) w(n-l) w(n-k+l)\right]\\ &= E\left[\sum_{k=0}^\infty \sum_{l=1}^{k-1}36\cdot 5^{-k} w(n-l) w(n-k+l)\right]\\ &= E\left[36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} w(n-l) w(n-k+l)\right]\\[2em] % % E\left[x^2(n)\right]&= \frac32 + 2E\left[\sum_{p=1}^\infty w(n)w(n-p)6\cdot5^{-p}\right] +E\left[36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} w(n-l) w(n-k+l)\right]\\ &= \frac32 + 12E\left[\sum_{p=1}^\infty w(n)w(n-p)\cdot5^{-p}\right] + 36 E\left[\sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} w(n-l) w(n-k+l)\right] \end{align}

At this point, things get very hairy: We can proceed if we allow ourselves to "pull" the $$E$$ operator into these infinite sums. But that only works if the result is finite, otherwise we'd be making a false statement.

Therefore, we must assume that $$\left\lvert\sum_{p=1}^\infty E\left[w(n)w(n-p)\cdot5^{-p}\right]\right\rvert<\infty$$. A gut feeling is that as long as $$w(n)w(n-p) < 5^p$$, we're fine, but bear in mind that this might not be the case for just any noise! An example of cases where that would not be the case is an exponentially decaying noisy oscillator of unknown phase with an offset. (due to the offset: non-zero mean, the exponential envelope gives us a bounded variance if we observe across all times, but looking back into the past for $$p\to\infty$$, we get infinitely many values $$>5^p$$.)

So!!!! Check! YOUR! Model for $$w(n)$$!111eleven!

If you trust you're not in one of the cases where this goes wrong (please, be sure):

\begin{align} E\left[x^2(n)\right]&= \frac32 \\ &\phantom= + 12\sum_{p=1}^\infty 5^{-p}E\left[w(n)w(n-p)\right]\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} E\left[w(n-l) w(n-k+l)\right] \end{align}

Standard trick: $$v(n) = w(n) - m_w$$, that makes $$v(n)$$ have a zero mean. Also, $$\sum_{k=0}^{\infty} 5^{-k}=\frac54$$:

\begin{align} &= \frac32 \\ &\phantom= + 12\sum_{p=1}^\infty 5^{-p}\left(E\left[m_w^2 + m_w(v(n)+v(n-p))+v(n)v(n-p)\right]\right)\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1}\left( E\left[m_w^2 + m_w(v(n-l)+v(n-k+l)) + v(n-l)v(n-k+l)\right]\right)\\ &= \frac32 \\ &\phantom= + 12\sum_{p=1}^\infty\ 5^{-p}\left(E\left[m_w^2\right] +E\left[ m_w(v(n)+v(n-p))\right]+E\left[v(n)v(n-p)\right]\right)\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1}\left( E\left[m_w^2\right] + E\left[m_w(v(n-l)+v(n-k+l))\right] + E\left[v(n-l)v(n-k+l)\right]\right)\\ % &= \frac32 +12\cdot\frac54 m_w^2 +36\cdot \frac54 m_w^2 \\ &\phantom= + 12m_w\sum_{p=1}^\infty\ 5^{-p}(\underbrace{E\left[v(n)\right]}_{=0}+\underbrace{E\left[v(n-p)\right]}_{=0})\\ &\phantom= + 12\sum_{p=1}^\infty\ 5^{-p}E\left[v(n)v(n-p)\right]\\ &\phantom= + 36 m_w\sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1}E[(v(n-l)]+E[v(n-k+l)] \\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} E[v(n-l)v(n-k+l)]\\ &= \frac32 +60 m_w^2 \\ &\phantom= + 12\sum_{p=1}^\infty\ 5^{-p}E\left[v(n)v(n-p)\right]\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} E[v(n-l)v(n-k+l)]\\ &= 16.5 \\ &\phantom= + 12\sum_{p=1}^\infty\ 5^{-p}E\left[v(n)v(n-p)\right]\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1}. E[v(n-l)v(n-k+l) \end{align}

That's quite a nice result, innit?

We can now look at a few special cases:

1. Different points in time of the "mean-removed" $$w(n)-m_w = v(n)$$ are uncorrelated: $$E[x^2(n)] = 16.5$$.
2. $$E[v(n)v(n-\tau)] = \alpha^k$$:
3. $$k<5$$: $$E[x^2(n)]$$ converges, value needs to be derived
4. $$k\ge5$$: $$E[x^2(n)]$$ diverges
• I think there's a problem with you're code. Code doesn't show math Nov 8, 2020 at 20:18
• Marcus Muller dude!!!! you're a legend!! Thank you very much! Nov 8, 2020 at 21:54

One straightforward way of solving this is to rewrite the process $$x[n]$$ only in terms of $$w[n]$$:

$$x[n]=\sum_{k=0}^{\infty}w[k]h[n-k]\tag{1}$$

where $$h[n]$$ is the impulse response of the system described by the given difference equation.

Once you have $$h[n]$$, and with the given assumptions on $$w[n]$$, the power of $$x[n]$$ can be calculated as

$$E\left\{x^2[n]\right\}=\left(\sum_{k=0}^{\infty}h[k]\right)^2m_w^2+\left(\sum_{k=0}^{\infty}h^2[k]\right)\sigma_w^2\tag{2}$$

Since this is a homework-style problem, I'll leave the derivation of $$h[n]$$ up to you, but if you do things right (and if I did things right), the result should be

$$E\left\{x^2[n]\right\}=\left(2.5\right)^2m_w^2+2.5\sigma_w^2\tag{3}$$

• I think that my answer is correct ( assuming mean is zero ) but I'm interested in your approach so would you mind showing what you got for $h[k].$ It's for learning purposes because I'm not clear on how to handle the unit impulse since it happens twice. If it happened once, that would be easy because then it's just an AR(1). Thanks. Nov 9, 2020 at 21:17
• @markleeds: You can use the Z-transform to compute the transfer function: $$H(z)=\frac{1+z^{-1}}{1-0.2z^{-1}}$$ The corresponding impulse response is $$h[n]=(0.2)^nu[n]+(0.2)^{n-1}u[n-1]$$ I think that the numbers in (3) are correct, yet they are different from yours. If you like you can check the results and I'll gladly correct my answer if it turns out there's an error somewhere. Nov 10, 2020 at 10:09
• I was more interested in the approach so thanks. things like z-transforms are not my "thing" so I will look at what you did closely and learn something for sure. As far as the answers go, I could have made some algebra mistake also. I'll try to do it over this weekend and include the mean and see what I get. But, it's neat to know two different approaches to the problem. thanks again. Nov 10, 2020 at 14:19
• Matt: As I said, I'll look closer when I have more time ( I'm moving so it's chaos here so may not be this weekend ) but it's so interesting because if one squares your $h[n]$ term, it looks like one gets something like $0.2^{(2n)} u[n]^2 + 0.2^{(2n-2)}u[n-1]^2$ which looks quite similar to my term immediately after where I write "So it is". So, we're definitely doing something quite similar. I'll study it at some point and get back hopefully with what causes the difference. Nov 10, 2020 at 14:30