1
$\begingroup$

When calculating the spectrum estimation of a signal $x(t)$ with random noise, why do we use ($E$ for expectation) $$E[|X(\omega)|^2]$$ but not $$E[|X(\omega)|] $$

If it is because the random noise will make $X(\omega)$ a random variable for any fixed $\omega$, then the estimation will vary from time to time, which might make no sense to take the right information of this data. Why the "square" could work? Doesn't it change with time?

Or there are something wrongs of my concept?

$\endgroup$
  • $\begingroup$ Because it's a power estimate. And power is proportional to magnitude square... The latter would just be a Fourier spectrum estimate, but the former is a power spectrum estimate... $\endgroup$ – Fat32 Jan 12 at 20:57
2
$\begingroup$

I disagree with aspects of other the given answers.

It depends on signal model that is assumed.

If $x(t)=s(t)+n(t)$ such that the probability density $x(t)$ is

$$ p(x(t))=\frac{1}{\sqrt{2\pi}\sigma} e^{(x(t)-s(t))^2/(2\sigma^2)} $$ then $$ E\{ x(t) \}= s(t) \quad \text{and} \quad \ne 0 \quad \text{even if the time average of} \; s(t)=0 $$ and $$ E\{ x(t)^2 \}= s(t)^2+\sigma^2 $$ This model is presented in Chapter 3 of

Van Trees, H. L. "Detection, Estimation, and Modulation Theory. Part 1-Detection, Estimation, and Linear Modulation Theory." (1968).

An unknown signal is not necessarily assumed to be random. A unknown deterministic signal is not ergodic. The time average and expectation are not the same.

One can have signal models where $s(t)$ is random and $E\{s(t)\}=0$.

A large portion of the spectral analysis literature assumes $E\{s(t)\}=0$ but not all.

which gets us back to your original question. The quantity

$$ E\{ x(t)^2 \}= s(t)^2+\sigma^2 $$

is a reason why the expected value of the magnitude squared is commonly assumed, power,deterministic or random, adds (if independent,and a bit more complicated if correlated).

A final comment, many signal models have both random and deterministic characteristics.

A Bayesian model can take the form. $$ p(x(t))=\frac{1}{\sqrt{2\pi}\sigma} e^{(x(t)-s(t))^2/(2\sigma^2)} p(s(t),\sigma) $$ or if appropriate, a model can take the form: $$ p(x(t))=\frac{1}{\sqrt{2\pi}\sigma} e^{(x(t)-As(t))^2/(2\sigma^2)} p(A) $$

The interesting thing in Signal Processing is that algorithms that may not make the"correct" assumptions, often work well. This often gives an Engineer a trade space in providing a solution where there are implementation constraints.

$\endgroup$
3
$\begingroup$

$\operatorname{E}[|X(\omega)|^2]$ has the useful quality that if $S$ represents the clean signal and $N$ represents the noise, and $\operatorname{E}[S] = 0,$ $\operatorname{E}[N] = 0,$ and $S$ and $N$ are independent, then it follows from:

$$X = S + N,$$

that:

$$\operatorname{E}[|X(\omega)|^2] = \operatorname{E}[|S(\omega)|^2] + \operatorname{E}[|N(\omega)|^2].$$

$\operatorname{E}[|X(\omega)|]$ does not have a similar quality. It is less convenient mathematically.

$\endgroup$
  • 1
    $\begingroup$ Nice! And most noise we model is zero-mean, in which case$E[\lvert N(\omega)\rvert^2] = \text{Var}[N]$, which is often very useful, because variance is something easy to estimate by statistics :) +1 $\endgroup$ – Marcus Müller Jan 12 at 12:56
2
$\begingroup$

Simply because that gives us a power spectral density, and that is usually more helpful than some amplitude density. But that's just a convention. We could just as well use $\sqrt{E\left[\lvert X(\omega)\rvert^2\right]}$.

Your considerations with randomness are off. The whole point of the expectation operator $E$ is to give you a non-random property of a random process.


Footnote:
because you've mentioned $X(\omega)$ directly in your question about randomness, let me comment on $E[X]$ itself. It's but loosely coupled to your question:

We could not use $E\left[X(\omega)\right]$, because for most practical signals carrying information, that'll always be $0$. Think about it: for example, a sine has mean 0, i.e. $E[\sin t]=0$, but it certainly is different from a constant $0$ or from white noise; but all these have $E=0$. Problem, especially in frequency analysis is, that we model any signal to be representable as sum of complex sinusoids, which all have, no matter how you scale them, $E[e^{j\omega t}]=0$, and thus, and due to linearity of the expectation operator $E[x = \sum_{n\ne 0}\alpha_n e^{j_n\omega t}+ \text{const.}]=0+\text{const.}$.

$\endgroup$
  • $\begingroup$ @OlliNiemitalo true, but I thought that paragraph was in order because Yui was directly referring to $X(\omega)$ in the part of the question about randomness. Re-reading this question, I'd agree, if anything, the explanation on $E[X]$ would best-case be "footnote material". Will rework my answer; thanks! $\endgroup$ – Marcus Müller Jan 12 at 12:52
  • 2
    $\begingroup$ E{s(t)}=s(t) if s(t) is deterministic, regardless if the time average is 0 $\endgroup$ – Stanley Pawlukiewicz Jan 12 at 14:52
1
$\begingroup$

Statistical estimation means: getting some compound and meaningful value from a bunch of data. In other words, if you have a series of values $x_\lambda$, you may seek one single $\bar{x}$ to represent them all. The latter should be evenly close to the different $x_\lambda$. So generally, you need a distance function $D$, and you try to evaluate whether there exists some $\bar{x}$ whose global distance $G$ with all $x_\lambda$ is the smallest (to be "more representative"). Mathematically, you will try to find:

$$ \bar{x} = \arg \min G_\lambda (D(x_\lambda-x))$$

i.e. a value than minimizes a combination (over all $x_\lambda$, $G_\lambda$) of each individual distance $D(x_\lambda-x)$.

This may sound a little abstract, but this is a way to write it precisely. If you choose $D$ to be the squared distance, and $G_\lambda$ to be the sum (or squares) over all $\lambda$ indices, you get:

$$ \bar{x} = \arg \min \sum_\lambda (x_\lambda-x)^2$$ whose minimum is simply... the classical average: $$ \bar{x} = \frac{1}{|\Lambda|}\sum_{\lambda\in \Lambda} x_\lambda$$ where $|\Lambda|$ denotes the cardinal of the set of $\lambda$ values. I wrote it with the sum symbol, but for continuous values of $\lambda$ you would use the integral similarly. This simple solution results from the fact that the global distance is convex, has a single solution, and the squared distance minimum can be obtained in an explicit way, since the derivative of a square yields linear equations, and the average is just a linear combination of terms. If you change $D$ with the absolute value, you cannot get an explicit formula in general, but the result is called the median. If now you choose $G_\lambda $ as the maximum instead of a sum, you get the midrange estimator.

Because there is a belief in energy conservation, because of energy preservation under orthogonality, and also (perhaps mainly?) because the sum of squared distances is one of the few global that give a tractable, sensitive, and easy to compute formula, a lot of estimates are "least-squares", and your example shows that well. Standard spectral estimation in performed in energy, or least-squares, under the orthogonal umbrella of the Fourier transform and its colleagues (Walsh-Hadamard, Discrete Cosine Transform, discrete wavelets, etc.).

Meanwhile, to increase estimator performance, especially to outlying data, there exists robust spectral estimators, for instance:

Robust $L$-estimation based forms of signal transforms and time-frequency representations, IEEE Transactions on Signal Processing, 2003:

The $L$-estimation based signal transforms and time-frequency (TF) representations are introduced by considering the corresponding minimization problems in the Huber (1981, 1998) estimation theory. The standard signal transforms follow as the maximum likelihood solutions for the Gaussian additive noise environment. For signals corrupted by an impulse noise, the median-based transforms produce robust estimates of the non-noisy signal transforms. When the input noise is a mixture of Gaussian and impulse noise, the $L$-estimation-based signal transforms can outperform other estimates. In quadratic and higher order TF analysis, the resulting noise is inherently a mixture of the Gaussian input noise and an impulse noise component. In this case, the L-estimation-based signal representations can produce the best results. These transforms and TF representations give the standard and the median-based forms as special cases. A procedure for parameter selection in the $L$-estimation is proposed. The theory is illustrated and checked numerically.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.