4
$\begingroup$

In many applications such as estimation theory, when we need to estimate a parameter then we usually consider in presence of white gaussian noise of zero mean and some standard deviation. During Maximum likelihood estimation, we also use this assumption. So, my question is -

  1. Do we consider noise to be uncorrelated or correlated in estimation?

  2. What is the difference between correlated and uncorrelated noise and its significance

  3. Why do we consider such a property of correlated or uncorrelated in estimation and when we say that measurement noise is gaussian.

$\endgroup$
  • $\begingroup$ If a zero-mean noise process is white, then all of its samples are jointly independent of one another (and therefore uncorrelated). So for #1, if the noise is white, then it's always uncorrelated. Apart from that, it's hard to figure out what you're asking. $\endgroup$ – Jason R Jun 5 '14 at 20:54
  • $\begingroup$ Thank you for your prompt attention. I am not sound in my knowledge of signals and transforms. So, plz pardon is I ask why the samples are independent is the noise process is zero-mean and white? I am from physics backgrnd where we deal with color noise which are not zero mean. Then, will color noise, which will not have a flat power spectrum (hence not white) be correlated?What is the relationship between zer0 mean, white property and uncorrelation? $\endgroup$ – Srishti M Jun 5 '14 at 20:59
  • $\begingroup$ Lastly, my major concern is in estimation techniques such as MLE do we consider noise to be correlated or uncorrelated and why (for whichever is true) do we assume this? $\endgroup$ – Srishti M Jun 5 '14 at 21:01
4
$\begingroup$

1) It depends on the application, but in general, both. In some cases we can whiten the noise, say in the following situation:

$$Y = x+ \eta\text{ where }\eta \sim N(0, \Sigma)$$

where we want to estimate $x \in \mathcal{A}$ for some set $\mathcal{A}$ from $Y$. In this case, we multiply both sides by the inverse of the square root of $\Sigma$, and we have an estimation problem where $\Sigma^{-1/2} x \in \Sigma^{-1/2} \mathcal{A}$.

2) Two random variables are said to be correlated if their covariance is non-zero. In the cause of jointly Gaussian random variables, uncorrelated is equivalent to independent.

3) Uncorrelated things are often easier to deal with since things decouple (for example, when estimating in the MMSE sense, uncorrelated is a form of orthogonality (perpendicularity, geometrically)). Its the next best thing to independence (and in the most important case, the jointly Gaussian case, they coincide). As for when things are Gaussian, one may justify things via central limit theorem or just by saying the answer is usually easy to find, and several estimators coincide in the jointly Gaussian case (MAP, MMSE, MMAE) [In the jointly Gaussian MMSE case, the estimator is the linear MMSE estimator, which is quite easy to calculate and only relies on first and second order moments, which makes life a lot easier instead of relying on higher order moments].

MLE is just a paradigm for non-random parameter estimation (where as in Bayesian settings such as MAP, MMSE, MMAE, the quantity you are trying to estimate is modeled as random). None of these methods impose a constraint of signal+additive noise for parameter estimation.

You should try reading a book or two on the topic.

$\endgroup$
  • $\begingroup$ If delta is a known signal, we could just subtract it out and reduce the problem down to the prior case, but the framework still holds directly. Chaotic phenomena aren't necessarily random - in the most basic case, they are deterministic phenomena with exponential divergence in phase space (see Strogatz' nonlinear dynamics and chaos, for example). As for the rest of the question, go read a book on the topic or at least whatever on CNX (cnx.org) or wikipedia or something. $\endgroup$ – Batman Jun 7 '14 at 2:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.