6
$\begingroup$

Let us say I have a row vector $X = [x_1, x_2, x_3, \ldots, x_n]$ and another row vector $Y = [y_1, y_2, y_3, \ldots, y_n]$. I want to check whether the two vectors are statistically independent or not.

Now two vectors are said to be statistically independent if $E(X Y) = E(X) E(Y)$, where $E$ is the expectation operator.

So how would I calculate the expectation value for each of the concerned vectors?

Thank you for your suggestions - I'm really new to statistics.

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ Your statement "Now two vectors are said to be statistically independent if E(X Y) = E(X) * E(Y), where E is the expectation operator." is incorrect on several counts. Unless your vectors are vectors of random variables, there is no expectation operator that you can use. If the vectors are vectors of random variables, then the vectors are independent if their joint density factors into the product of the marginal densities: $$f_{X,Y}(u_1,\cdots,u_n,v_1,\cdots,v_n) = f_X(u_1,\cdots,u_n)f_Y(v_1,\cdots,v_n)$$ and the random variables are independent if the RHS factors into $2n$ densities. $\endgroup$
    – Dilip Sarwate
    Commented Feb 27, 2012 at 14:52
  • 1
    $\begingroup$ Since you asked about doing this in MATLAB, then I'll assume that you have two vectors of "numbers" that are really observations from two random processes. In that case, there's no single operation that you can do in order to declare the two underlying processes as statistically independent. You need to at least have some probability model for the sources of the realizations that you're looking at. There are then statistical tests that you can apply to determine the amount of dependence with some amount of confidence. $\endgroup$
    – Jason R
    Commented Feb 27, 2012 at 15:05
  • $\begingroup$ @DilipSarwate Thank you for your comment. So if I have for example two vectors - one vector is sin(x), and the other vector has numbers drawn from a Gaussian distribution.. You mentioned that they HAVE to be vectors of random variables. So sine wouldn't count? Also, if they are random, how would you compute the joint density factors. I've really been looking into this but can't seem to find a straight answer. $\endgroup$
    – Rachel
    Commented Feb 27, 2012 at 15:56
  • 2
    $\begingroup$ @DilipSarwate Joint PDF being a product of the marginals is correct, but I do not think this helps the OP. She is asking about a test. Let us assume for the sake of argument that the two vectors were indeed made by 2 random variables. From HERE, how do we test for independence? $\endgroup$
    – Spacey
    Commented Feb 27, 2012 at 16:59
  • 1
    $\begingroup$ @Rachel If one of your vectors is deterministic as in $$X=[\sin(\omega t_1),\sin(\omega t_2),\ldots,\sin(\omega t_n))$$ where the $t_i$ are fixed time instants (not necessarily equally spaced in time) and $\omega$ is a known constant, and $Y$ is a vector obtained by calling randn (say), then $X$ and $Y$ are independent in that knowing one tells you nothing about the other, and this is reflected in the probability assignment. If $X$ and $Y$ are generated by separate calls to randn and you are not playing tricks by forcing the same "seed" in both calls, then $X$ and $Y$ are independent. $\endgroup$
    – Dilip Sarwate
    Commented Feb 28, 2012 at 3:35

1 Answer 1

Reset to default
1
$\begingroup$

First off, I would read Examples of Independent and uncorrelated data in real-life, and ways to measure/detect them.
It will likely prove useful to you.

If you are looking for something quick, easy, and useful, I would simply cross-correlate the two vectors and normalize the result. A small result (less that 0.05?) indicates, but does not prove, independence. A large result (more than 0.2?) would indicate some amount of dependence. If the vectors have a non-zero mean, you may need to subtract out the mean before doing the correlation, depending on why they are non-zero mean.

You could also go a more academic route and use the method pichenettes outlines (estimate the probability distribution using a kernel density estimator) in the thread that I linked to at the beginning of the post.

Improve this answer
$\endgroup$
5
  • $\begingroup$ First of all thank you. I had in fact already read that thread and have a similar open question at dsp.stackexchange.com/questions/1523/…. Can you please give an example of how to calculate the cross-correlation value? And does this expand to n vectors? $\endgroup$
    – Rachel
    Commented Feb 27, 2012 at 20:13
  • $\begingroup$ @Rachel The mathematical explanation of how to do it can be found at the Wikipedia link above. The simple explanation is that for the zero-offset cross correlation you simply multiply element 1 in X and Y, multiply element 2 in X and Y, ..., and sum the products. The result is normalized by dividing by the square root of the power of both X and Y. Rather than try to set all that up, though, I would just use Matlab's "xcorr" command. $\endgroup$
    – Jim Clay
    Commented Feb 27, 2012 at 20:19
  • $\begingroup$ @Rachel Regarding scaling it up to "n" vectors- yes and no. You can do it, but you have to cross-correlate all possible pairs of the n vectors, so it can get ugly computationally as n gets large. $\endgroup$
    – Jim Clay
    Commented Feb 27, 2012 at 20:20
  • $\begingroup$ @JimClay I am somewhat suspicious of the correlation test here ... cross-correlation gives us a score of how correlated two vectors are, but says nothing about how dependent they might be? (Example, you get a correlation score of 0, but they might still be dependent). Isnt this true? If yes, they how can we use it as a sort of test? $\endgroup$
    – Spacey
    Commented Feb 29, 2012 at 1:14
  • $\begingroup$ @Mohammad Yes, there can be zero correlation even if the signals are dependent- particularly since we are only looking at a finite time window. That is why I have said on other threads that non-correlation is indicative of independence, but not proof. $\endgroup$
    – Jim Clay
    Commented Feb 29, 2012 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.