4
$\begingroup$

I want to know the difference between independent and identically distributed (i.i.d) noise and white noise.

In my short knowledge, i.i.d is that there is no relationship about time dependancy. White noise means that there are relationship about time dependancy.

Actually, I'm not sure whether this is correct or not. Also I want to know what is an iid white noise.

Can you tell me where we find the iid noise in the nature?

update

enter image description here enter image description here

$\endgroup$

closed as unclear what you're asking by jojek, lennon310, Peter K. Jun 5 '15 at 13:00

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

6
$\begingroup$

Independence is 'stronger' than whiteness. I believe that independence between the random variables implies whiteness but whiteness does not imply independence. Whiteness means that the random variables are uncorrelated but not necessarily independent.

The use of i.i.d noise is seen very often when formulating probabilistic models because it makes inference much easier. For instance if two random variables $X_1$ and $X_2$ are i.i.d it means that the joint pdf $p(X_1,X_2)$ factors into the product of the individual pdfs $p(X_1)p(X_2)$. If the two random variables are uncorrelated this factorization is not valid.

In ML estimation typically the log of the product is considered $\log(p(X_1)p(X_2)) = \sum \log p(X)$ because then differentiation with respect to the parameter of interest is much more straight forward.

I'm not sure where to find i.i.d noise in the real world but I believe that the assumption about i.i.d observation noise is made more of convenience than because it is realistic.

$\endgroup$
  • 1
    $\begingroup$ It should also be noted that for Gaussian random variables, the terms uncorrelated and independent are synonomous; if two Gaussian random variables are uncorrelated, then they are also independent. This is a particular feature of the Gaussian distribution, and isn't generally applicable. So the terms iid and white are similar when the underlying distribution is Gaussian (with the caveat that iid implies that all of the random variables are identically distributed, which is stronger than just saying that all of the random variables are jointly uncorrelated). $\endgroup$ – Jason R Jun 5 '15 at 15:05
  • $\begingroup$ @Jason R Thanks,Then can you tell me the following questions of mine? In the answer, In ML estimation typically the log of the product is considered log(p(X1)p(X2))=∑logp(X).Then can I tell like this that typically the product is considered (p(X1)p(X2))=∑p(X). If yes, can I also write like this ? (p(X1)p(X2))= p(X1)+p(X2). (I just rephrased above equation because to find whether my idea correct or not). $\endgroup$ – gmotree Jun 5 '15 at 16:30
  • 1
    $\begingroup$ I don't follow how you got from the first equation there to the second. The first relies upon a property of the logarithm, namely that $\log{xy} = \log{x} + \log{y}$. $\endgroup$ – Jason R Jun 5 '15 at 16:34
  • 3
    $\begingroup$ @JasonR Your statement "It should also be noted that for Gaussian random variables, the terms uncorrelated and independent are synonomous; if two Gaussian random variables are uncorrelated, then they are also independent." is incorrect. The standard counter-example is $X \sim N(0,1)$, $Z$ independent of $X$ and equally likely to take on values $+1$ and $-1$, and $Y = ZX$. It is straightforward to verify that $Y \sim N(0,1)$ while $$\operatorname{cov}(X,Y) = E[XY] - E[X]E[Y]= E[X^2Z]=E[X^2]E[Z] = 0.$$ $\endgroup$ – Dilip Sarwate Jun 5 '15 at 18:40
  • 1
    $\begingroup$ @DilipSarwate: Indeed, you're right. This is a popular enough misconception that there is a Wikipedia article about it. Only a pair of jointly Gaussian random variables has the property that I referenced. $\endgroup$ – Jason R Jun 5 '15 at 18:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.