What is definition of independent random variable

Question

I wan't to ask that if

E{X}=0

E{Y}=0

and E{XY}=0

then how can I verify if the two random variables are independent or not.

X , Y are both continuous random variables {I am not able to recall the exact equations for X , Y but I am sure about the information given above and want some hint of how to proceed}.

What I know about independence {the definition that is causing confusion}

And so E{XY} should be = E{X}E{Y}

Which part of the below derivation doesn't sufficiently satisfies that X and Y are independent {where did it went wrong}

Transition from first line to second line is only possible if two random variables are independent and after it it's just rearrangement of terms.

Answer 1

No, NO and $\mathbf{NO}$.

Knowing that $E[X]=E[Y]=0$ and $E[XY]=0$ (or more generally that $E[XY]=E[X]E[Y]$ or equivalently, $E[XY]-E[X]E[Y]=0$) does not help in the least in proving or deducing that $X$ and $Y$ are independent random variables. The conditions stated in the previous sentence are necessary for $X$ and $Y$ to be deemed independent random variables, but they are a very far cry from being sufficient for independence to hold. That is, if $E[XY]=E[X]E[Y]$, then we might suspect that $X$ and $Y$ might be independent random variables, but we cannot jump to the conclusion that they are independent random variables based on just the evidence that $E[XY]=E[X]E[Y]$.

Definition: $X$ and $Y$ are said to be independent random variables (sometimes more prolixly mutually independent random variables) if their joint distribution function (joint CDF) $F_{X,Y}(u,v) = P\{X \leq u, Y\leq v\}$ satisfies $$F_{X,Y}(u,v)=F_{X}(u)F_{Y}(v) ~\text{for all real numbers}~ u ~\text{and}~v.\tag{1}$$ Here, $F_{X}(u)=P\{X \leq u\}$ and $F_{Y}(v)=P\{Y\leq v\}$ are the marginal CDFs of $X$ and $Y$ respectively. It is important to remember the qualifier to the equality $F_{X,Y}(u,v)=F_{X}(u)F_{Y}(v)$: it must hold at all points in the $u$-$v$ plane.. Inanities such as "$X$ and $Y$ are independent when $X=5$ and $Y=3$ but not otherwise" are not permitted.

If $X$ and $Y$ are discrete random variables, then $(1)$ is equivalent to their joint _probability mass function (joint pmf) $p_{X,Y}(u_i, v_j)$ factoring into the product $p_X(u_i)p_Y(v_j)$ of the marginal pdfs for all $i$ and $j$.

If $X$ and $Y$ are jointly continuous random variables, then $(1)$ is equivalent to their joint _probability density function (joint pdf) $f_{X,Y}(u, v)$ factoring into the product $f_X(u)f_Y(v)$ of the marginal pdfs _for all real numbers $u$ and $v$.

Now, as the OP correctly points out, if $X$ and $Y$ are independent, then it is true that $E[XY]=E[X]E[Y]$, that is, $$X, Y ~\text{independent random variables} \implies E[XY]=E[X]E[Y]$$ but

the reverse implication $$E[XY]=E[X]E[Y]\implies X, Y ~\text{independent random variables}$$ $$\mathbf{DOES~NOT~HOLD}$$

Still don't believe a word of all this malarkey? Consider discrete random variables $X$ and $Y$ such that $$p_{X,Y}(1,0) = p_{X,Y}(-1,0) = p_{X,Y}(0,1) = p_{X,Y}(0,-1) = \frac 14.$$ Verify that

1. $X$ and $Y$ take on values $\pm 1$ with equal probability $\frac 14$ and value $0$ with probability $\frac 12$.
2. $XY=0$ always and so $E[XY]=0$ also, and that $E[X]=E[Y]=0$ also.
3. The joint pmf does not factor into the product of the marginal pdfs for at any of the four points where the joint pmf is nonzero.

Answer 2

Essentially $E\{XY\}=0$ (assuming $E\{X\}=E\{Y\}=0$) means that $X$ and $Y$ are uncorrelated, but uncorrelated is a weaker property than probabilistic independence. A collection of independent random variables will always be uncorrelated, but being uncorrelated does not mean two random variables are independent.

Answer 3

Let's look at things in the following perspective;

Statistical independence is an assumption imposed on events $A,B$ or random variables $X,Y$, whose (the assumption's) mathematical consequence is expressed as; $$ P( A \cap B) = P(A) P(B)$$ and $$ E\{X Y\} = E\{X\}E\{Y\} $$

So these are the consequences of independence, but not the cause of it. (This is what Dilip Sarwate expresses in his answer).

There's a third consequence of (and is also a cause for) independence, regarding joint and marginal CDF/PDFs:

$$ F_{XY}(x,y) = F_X(x) F_Y(y) .$$ This will hold for all $x,y$ only when $X$ and $Y$ are independent.

Furthermore, if this equality can be shown to hold for all $x,y$ (without assuming $X$ and $Y$ as independent) then from this we can also conclude that the random variables $X$ and $Y$ are independent. Hence both a consequence of and a cause for it.

The above thing is mainly theoretical argumentation. In practice, from observed practical data, you can never fully and precisely check the third consequence and hence can never rigorously prove that random variables are indeed independent. This is not the preferred approach.

What you can practically do is to check (actually estimate) for the following to hold.

$$\sigma_{XY} = E\{ (X-\mu_Y)(Y-\mu_Y) \} = 0 $$ which implies that $$ E\{XY\} = E\{X\} E\{Y\} $$ and conclude that $X$ and $Y$ are uncorrelated. (This is what RBJ expresses in his answer.) Note that when X and Y are assumed to be independent, then as a consequence, (by the last equation), they will also be uncorrelated; but the converse is not true as explained in the first part.

Again note that this is an estimation and is never perfectly accurate. Hence one can never be truly sure that RVs $X$ and $Y$ are indeed uncorrelated or be sure that whether the result of a normalized correlation being $0.001$ indicates an innacurate computation or just a slight correlation between $X$ and $Y$... What can only be said is that a slight correlation and truly uncorrelatedness will have the same (almost indistinguishable) effects on the result of DSP algorithms.

Answer 4

The relationship $\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[X]$ says that the two random variables are uncorrelated. Uncorrelated random variables are in general not independent, unless they are Gaussian random variables.