Noise added to a Random Process

Question

if we have a discrete random process \begin{equation} x\left(n\right)\:=\:0.2x\left(n-1\right)+w\left(n\right)+w\left(n-1\right) \end{equation} where $ w\left(n\right)$ is a noise with a mean $ m_w=0.5$ and variance $\sigma ^2_w\:=\:1$.

I am trying to find the value of $E\left\{x^2\left(n\right)\right\}$.

Here's my work below:

\begin{equation} E\left\{x^2\left(n\right)\right\} = E\bigg\{\big[0.2x\left(n-1\right)+w\left(n\right)+w\left(n-1\right)\big]^2\bigg\} \end{equation}

for $$E\big\{w\left(n-i\right)^2\big\}=\sigma _w^2+m_w^2=1+0.5\:=\:1.5.$$

After that, I get the following \begin{align} E\left\{x^2\left(n\right)\right\} =&\phantom{+} 0.04 E\left\{x^2\left(n-1\right)\right\}\\ &+0.4E\left\{w\left(n\right)x\left(n-1\right)\right\}\\ &+0.4E\left\{w\left(n-1\right)x\left(n-1\right)\right\}\\ &+E\left\{w^2\left(n\right)\right\}\\ &+2E\left\{w\left(n\right)w\left(n-1\right)\right\}\\ &+E\left\{w^2\left(n-1\right)\right\} \end{align}

Am not sure what to do from here? Please help.

Answer 1

Hi: I don't know whether white noise can have a non-zero mean ( Dilipe can tell us that ) but, if it can't, then just call it whatever you call white noise with a non-zero mean.

But, assuming it's zero, this is how you can do it. ( If it's not zero, then the solution is similar but the last step would be a little different. Use $$\operatorname{Var}\left\{X\right\} = E\left\{X^2\right\} - E\left\{X\right\}^2$$ Note also I wrote it with "t's" and "epsilon's" just because I'm more used to those but you can think of $t$ as $n$ and $\epsilon$ as $w$.

\begin{align} x_t &= 0.2 x_{t-1} + \epsilon_{t} + \epsilon_{t-1}\\ \implies x_{t}(1 - 0.2L) &= \epsilon_{t} + \epsilon_{t-1} \tag{$\scriptsize{\text{using lag operator}}$}\\ \implies x_{t} &= \frac{\epsilon_{t}}{1 - 0.2L} + \frac{\epsilon_{t-1}}{1 - 0.2L}\\ \implies x_{t} &= \sum_{i = 0}^{\infty} 0.2^{i} \epsilon_{t-i} + \sum_{i = 0}^{\infty} 0.2^{i} \epsilon_{t-1-i} \end{align}

So, since the mean is zero, the $E\left\{x^2_{t}\right\}$ is the variance of the RHS.

So, it is \begin{align} \sum_{i = 0}^{\infty} \left(0.2^{i}\right)^2 \cdot \sigma^2_\epsilon + \sum_{i = 0}^{\infty} \left(0.2^{i}\right)^2 \cdot \sigma^2_\epsilon&= \sum_{i = 0}^{\infty} 0.04^{i} \cdot \sigma^2_\epsilon + \sum_{i = 0}^{\infty} 0.04^{i} \cdot \sigma^2_\epsilon\\ &= \frac{2 \cdot \sigma^2_{\epsilon}}{(1- 0.04)} \end{align}

Answer 2

Let's define a helper: $y(n) = w(n) + w(n-1)$. Then, your $x$ becomes

\begin{align} x(n) &= \frac15 x(n-1) + y(n)\\ &= \frac15 \left(\frac 15x(n-2)+y(n-1)\right) + y(n)\\ &=\frac 1{25}x(n-2)+\frac15 y(n-1)+ y(n)\\ &=\frac1{5^3}x(n-3)+\frac 1{25}y(n-2)+\frac15 y(n-1)+ y(n)\\ &=\sum_{p=0}^\infty 5^{-p}y(n-p)\\ &=\sum_{p=0}^\infty\left( 5^{-p}w(n-p)+5^{-p}w(n-p-1)\right)\\ &=\sum_{p=0}^\infty 5^{-p}w(n-p)+\sum_{q=1}^\infty5^{-q+1}w(n-q)\\ &=5^{-0}w(n)+\sum_{p=1}^\infty 5^{-p}w(n-p)+\sum_{q=1}^\infty5^{-q+1}w(n-q)\\ &=w(n)+\sum_{p=1}^\infty \left(5^{-p}w(n-p)+5^{-p+1}w(n-p)\right)\\ &=w(n)+\sum_{p=1}^\infty w(n-p)\left(5^{-p}+5^{-p+1}\right)\\ &=w(n)+6\sum_{p=1}^\infty w(n-p)5^{-p}\\[2em] % % E\left[x^2(n)\right] &= E\Big[\Big(w(n) +\underbrace{\sum_{p=1}^\infty w(n-p)6\cdot5^{-p}}_{s(n)}\Big)^2\Big]\\ &= E[w^2(n)] + E[2w(n)s(n)] + E[s^2(n)]\\ &= \frac32 + 2E[w(n)s(n)] + E[s^2(n)]\\ &= \frac32 + 2E\left[\sum_{p=1}^\infty w(n)w(n-p)6\cdot5^{-p}\right] + E[s^2(n)]\\[2.5em] % % E[s^2(n)] &= E[s(n)\cdot s(n)]\\ &\text{Cauchy product formula:}\\ &= E\left[\sum_{k=0}^\infty\sum_{l=1}^{k-1} w(n-l)\left(5^{-l}+5^{-l+1}\right) w(n-k+l)\left(5^{-k+l}+5^{-k+l+1}\right)\right]\\ &= E\left[\sum_{k=0}^\infty \sum_{l=1}^{k-1}\left(5^{-l}+5^{-l+1}\right)\left(5^{-k+l}+5^{-k+l+1}\right) w(n-l) w(n-k+l)\right]\\ &= E\left[\sum_{k=0}^\infty \sum_{l=1}^{k-1}36\cdot 5^{-k} w(n-l) w(n-k+l)\right]\\ &= E\left[36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} w(n-l) w(n-k+l)\right]\\[2em] % % E\left[x^2(n)\right]&= \frac32 + 2E\left[\sum_{p=1}^\infty w(n)w(n-p)6\cdot5^{-p}\right] +E\left[36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} w(n-l) w(n-k+l)\right]\\ &= \frac32 + 12E\left[\sum_{p=1}^\infty w(n)w(n-p)\cdot5^{-p}\right] + 36 E\left[\sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} w(n-l) w(n-k+l)\right] \end{align}

At this point, things get very hairy: We can proceed if we allow ourselves to "pull" the $E$ operator into these infinite sums. But that only works if the result is finite, otherwise we'd be making a false statement.

Therefore, we must assume that $\left\lvert\sum_{p=1}^\infty E\left[w(n)w(n-p)\cdot5^{-p}\right]\right\rvert<\infty$. A gut feeling is that as long as $w(n)w(n-p) < 5^p$, we're fine, but bear in mind that this might not be the case for just any noise! An example of cases where that would not be the case is an exponentially decaying noisy oscillator of unknown phase with an offset. (due to the offset: non-zero mean, the exponential envelope gives us a bounded variance if we observe across all times, but looking back into the past for $p\to\infty$, we get infinitely many values $>5^p$.)

So!!!! Check! YOUR! Model for $w(n)$!111eleven!

If you trust you're not in one of the cases where this goes wrong (please, be sure):

\begin{align} E\left[x^2(n)\right]&= \frac32 \\ &\phantom= + 12\sum_{p=1}^\infty 5^{-p}E\left[w(n)w(n-p)\right]\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} E\left[w(n-l) w(n-k+l)\right] \end{align}

Standard trick: $v(n) = w(n) - m_w$, that makes $v(n)$ have a zero mean. Also, $\sum_{k=0}^{\infty} 5^{-k}=\frac54$:

\begin{align} &= \frac32 \\ &\phantom= + 12\sum_{p=1}^\infty 5^{-p}\left(E\left[m_w^2 + m_w(v(n)+v(n-p))+v(n)v(n-p)\right]\right)\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1}\left( E\left[m_w^2 + m_w(v(n-l)+v(n-k+l)) + v(n-l)v(n-k+l)\right]\right)\\ &= \frac32 \\ &\phantom= + 12\sum_{p=1}^\infty\ 5^{-p}\left(E\left[m_w^2\right] +E\left[ m_w(v(n)+v(n-p))\right]+E\left[v(n)v(n-p)\right]\right)\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1}\left( E\left[m_w^2\right] + E\left[m_w(v(n-l)+v(n-k+l))\right] + E\left[v(n-l)v(n-k+l)\right]\right)\\ % &= \frac32 +12\cdot\frac54 m_w^2 +36\cdot \frac54 m_w^2 \\ &\phantom= + 12m_w\sum_{p=1}^\infty\ 5^{-p}(\underbrace{E\left[v(n)\right]}_{=0}+\underbrace{E\left[v(n-p)\right]}_{=0})\\ &\phantom= + 12\sum_{p=1}^\infty\ 5^{-p}E\left[v(n)v(n-p)\right]\\ &\phantom= + 36 m_w\sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1}E[(v(n-l)]+E[v(n-k+l)] \\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} E[v(n-l)v(n-k+l)]\\ &= \frac32 +60 m_w^2 \\ &\phantom= + 12\sum_{p=1}^\infty\ 5^{-p}E\left[v(n)v(n-p)\right]\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1} E[v(n-l)v(n-k+l)]\\ &= 16.5 \\ &\phantom= + 12\sum_{p=1}^\infty\ 5^{-p}E\left[v(n)v(n-p)\right]\\ &\phantom= + 36 \sum_{k=0}^\infty 5^{-k} \sum_{l=1}^{k-1}. E[v(n-l)v(n-k+l) \end{align}

That's quite a nice result, innit?

We can now look at a few special cases:

1. Different points in time of the "mean-removed" $w(n)-m_w = v(n)$ are uncorrelated: $E[x^2(n)] = 16.5$.
2. $E[v(n)v(n-\tau)] = \alpha^k$:
3. $k<5$: $E[x^2(n)]$ converges, value needs to be derived
4. $k\ge5$: $E[x^2(n)]$ diverges

Answer 3

One straightforward way of solving this is to rewrite the process $x[n]$ only in terms of $w[n]$:

$$x[n]=\sum_{k=0}^{\infty}w[k]h[n-k]\tag{1}$$

where $h[n]$ is the impulse response of the system described by the given difference equation.

Once you have $h[n]$, and with the given assumptions on $w[n]$, the power of $x[n]$ can be calculated as

$$E\left\{x^2[n]\right\}=\left(\sum_{k=0}^{\infty}h[k]\right)^2m_w^2+\left(\sum_{k=0}^{\infty}h^2[k]\right)\sigma_w^2\tag{2}$$

Since this is a homework-style problem, I'll leave the derivation of $h[n]$ up to you, but if you do things right (and if I did things right), the result should be

$$E\left\{x^2[n]\right\}=\left(2.5\right)^2m_w^2+2.5\sigma_w^2\tag{3}$$