# Problem understanding the Expectation Operator

I know that the Expectation Operator $$E\{x\}$$ four discrete values is $$\sum_k \alpha Pr(x = \alpha_k)$$

and its very intuitive when speaking out a formula which contains the Expectation Operator. But I often have some troubles when trying to apply it on real examples with numbers.

E.g. the autocorrelation is completely obvious when given as sum like: $$r_x(\tau)=\sum_n x_n x_{n-\tau}.$$ However, often principles like autocorrelation, correlation, cross-correlation etc. are formulated with the $$E$$ Operator. For the autocorrelation: $$r_x(k,l) = E\{x(k)x^*(l)\}.$$ I assume this can be written as: $$r_x(\tau) = E\{x(k)x^*(k-\tau)\}.$$

The product is the same as in the sum above. However, the $$E$$ Operator implies that I need to know some probabilities. How can one quickly see that this is the same as above with $$\alpha$$ is somehow $$x_n x_{n-\tau}$$ and $$Pr=1$$? This can't always be the case. Otherwise one could always write the sum instead of the E Operator, couldn't he?

I think about the E Operator as if it is needed for different random processes. This would imply that I can't formulate e.g. a cross-correlation as a sum over all products between two signals, as I used to do. A formula like $$r_{xy}(k,l)=E\{x(k)y^*(l)\}$$ would then be a more general case and could not be applied on a simple numerical example, without making assumptions about the processes statistical properties.

• it seems to me that your root probem is the transition from the random variables to random process ? – Fat32 Oct 28 '18 at 10:57
• yes I guess thats at least a part of my problem. – Mr.Sh4nnon Oct 28 '18 at 17:41

Consider a discrete-time, discrete valued random process $$X[n,s)$$ whose samples $$x[n]$$ at integer index $$n$$ belong to range space $$R_{X_n} = \{x_1,x_2,...,x_n\}$$ of discrete valued random variables $$X_n$$ indexed by $$n$$, such that at each $$n$$, the value $$x[n]$$ is the result of the mapping by $$X_n$$ into $$R_{X_n}$$; i.e., $$X[n,s) = X_n(s) = x[n]$$, where $$s$$ is the sample output of the experiment performed at time $$n$$.

The expectetation of a particular random variable $$X_n$$ of the random process is $$\mathcal{E} \{ X_n \} = \sum_{x_k \in R_{X_n}} x_k P_{X_n}(x_k) = \int_{-\infty}^{\infty} x f_{X_n}(x) dx$$ where those $$x_k$$ belong to the range space of the random variable $$X_n$$ and $$P_{X_n}(x_k)$$ are the probabilities of those values. Note that the integral definition is also available for those who would still like to use a continuous random variable approach with impulsive PDFs; $$f_{X_n}(x) = \sum_k P_{X_n}(x_k) \delta(x-x_k)$$

The correlation between two (real) random variables $$X_n$$ and $$X_m$$ is defined as $$\mathcal{E} \{ X_n X_m \} = \sum_{k} \sum_{l} x_k x_l P_{X_n,X_m}(x_k,x_l)$$ where $$P_{X_n,X_m}(x_k,x_l)$$ is the joint probability mass function of the two random variables, and signifies the probability of the pair $$(x_k,x_l)$$ in the range space of the joint mapping.

When those two random variables belong to the same random process, then the correlation is defined as auto-correlation of the random process.

Note that the second sum in your post does not represent an expectation but rather a possibly ergodic estimate of it for a WSS random process.

• Fat32's answer was fine but, for emphasis, I think you're missing or confusing the difference between an estimate and its expectation. The second equation is an estimate ( assuming the r's were estimates ) of the expectation. In general, the expectations are hardly ever known and, by assuming ergodicity, one estimates these expectations by using the estimates from that process. Also, the "I think this can be written" part is only true if the series is covariance stationary. but, in any case, you are calculating an expectation there rather than an estimate of it. – mark leeds Oct 28 '18 at 15:15
• So this means, in the first years of engineering at university everyone assumes that $P_{X_n,X_m}(x_k,x_l)$ is 1 for the autocorrelation due to a WSS random process? My assumption that writing stuff with the Expectation operator makes stuff generally more applicable is than correct, right? I guess I have to look up which other formulas I used earlier are just estimations. – Mr.Sh4nnon Oct 28 '18 at 17:50
• Also as the @markleeds comment indicates (and last line of my answer too) please differentiate between the theoretical, probabilistic-statistical, expectation operation $$\mu_x = E\{X\} = \int_{-\infty}^{\infty} x f_X(x) dx$$ from its numerical, practical, Estimate $$\hat{\mu}_X = < x> = \frac{1}{N} \sum_n x_n$$ based on ergodocity assumption of the observation. Which means that ensemble averages can be replaced with time averages (for a random process) – Fat32 Oct 28 '18 at 17:54
• @Mr.Sh4nnon to understand the joint pdf between two random variables, just consult a standart (engineering) probability textbook on the chapter multiple random variables. $P_{X_n,X_m}(x_k,x_l)$ is not assumed to be one. If it were 1, then there is only one elemnt of the range space, which would be the certain element with probability 1. – Fat32 Oct 28 '18 at 17:57
• @Fat32 thank you very much. You're last two comments are completely clear. Already knew that. I just realised that some formulas which they thought us in the second year is only valid under some assumptions. Now that we are doing more advanced signal processing this caused some confusion... – Mr.Sh4nnon Oct 28 '18 at 18:03