5
$\begingroup$

Let us say I have a row vector $X = [x_1, x_2, x_3, \ldots, x_n]$ and another row vector $Y = [y_1, y_2, y_3, \ldots, y_n]$. I want to check whether the two vectors are statistically independent or not.

Now two vectors are said to be statistically independent if $E(X Y) = E(X) E(Y)$, where $E$ is the expectation operator.

So how would I calculate the expectation value for each of the concerned vectors?

Thank you for your suggestions - I'm really new to statistics.

$\endgroup$
  • 2
    $\begingroup$ Your statement "Now two vectors are said to be statistically independent if E(X Y) = E(X) * E(Y), where E is the expectation operator." is incorrect on several counts. Unless your vectors are vectors of random variables, there is no expectation operator that you can use. If the vectors are vectors of random variables, then the vectors are independent if their joint density factors into the product of the marginal densities: $$f_{X,Y}(u_1,\cdots,u_n,v_1,\cdots,v_n) = f_X(u_1,\cdots,u_n)f_Y(v_1,\cdots,v_n)$$ and the random variables are independent if the RHS factors into $2n$ densities. $\endgroup$ – Dilip Sarwate Feb 27 '12 at 14:52
  • 1
    $\begingroup$ Since you asked about doing this in MATLAB, then I'll assume that you have two vectors of "numbers" that are really observations from two random processes. In that case, there's no single operation that you can do in order to declare the two underlying processes as statistically independent. You need to at least have some probability model for the sources of the realizations that you're looking at. There are then statistical tests that you can apply to determine the amount of dependence with some amount of confidence. $\endgroup$ – Jason R Feb 27 '12 at 15:05
  • $\begingroup$ @DilipSarwate Thank you for your comment. So if I have for example two vectors - one vector is sin(x), and the other vector has numbers drawn from a Gaussian distribution.. You mentioned that they HAVE to be vectors of random variables. So sine wouldn't count? Also, if they are random, how would you compute the joint density factors. I've really been looking into this but can't seem to find a straight answer. $\endgroup$ – Rachel Feb 27 '12 at 15:56
  • 2
    $\begingroup$ @DilipSarwate Joint PDF being a product of the marginals is correct, but I do not think this helps the OP. She is asking about a test. Let us assume for the sake of argument that the two vectors were indeed made by 2 random variables. From HERE, how do we test for independence? $\endgroup$ – Spacey Feb 27 '12 at 16:59
  • 1
    $\begingroup$ @Rachel If one of your vectors is deterministic as in $$X=[\sin(\omega t_1),\sin(\omega t_2),\ldots,\sin(\omega t_n))$$ where the $t_i$ are fixed time instants (not necessarily equally spaced in time) and $\omega$ is a known constant, and $Y$ is a vector obtained by calling randn (say), then $X$ and $Y$ are independent in that knowing one tells you nothing about the other, and this is reflected in the probability assignment. If $X$ and $Y$ are generated by separate calls to randn and you are not playing tricks by forcing the same "seed" in both calls, then $X$ and $Y$ are independent. $\endgroup$ – Dilip Sarwate Feb 28 '12 at 3:35
1
$\begingroup$

First off, I would read this thread. It will likely prove useful to you.

If you are looking for something quick, easy, and useful, I would simply cross-correlate the two vectors and normalize the result. A small result (less that 0.05?) indicates, but does not prove, independence. A large result (more than 0.2?) would indicate some amount of dependence. If the vectors have a non-zero mean, you may need to subtract out the mean before doing the correlation, depending on why they are non-zero mean.

You could also go a more academic route and use the method pichenettes outlines (estimate the probability distribution using a kernel density estimator) in the thread that I linked to at the beginning of the post.

$\endgroup$
  • $\begingroup$ First of all thank you. I had in fact already read that thread and have a similar open question at dsp.stackexchange.com/questions/1523/…. Can you please give an example of how to calculate the cross-correlation value? And does this expand to n vectors? $\endgroup$ – Rachel Feb 27 '12 at 20:13
  • $\begingroup$ @Rachel The mathematical explanation of how to do it can be found at the Wikipedia link above. The simple explanation is that for the zero-offset cross correlation you simply multiply element 1 in X and Y, multiply element 2 in X and Y, ..., and sum the products. The result is normalized by dividing by the square root of the power of both X and Y. Rather than try to set all that up, though, I would just use Matlab's "xcorr" command. $\endgroup$ – Jim Clay Feb 27 '12 at 20:19
  • $\begingroup$ @Rachel Regarding scaling it up to "n" vectors- yes and no. You can do it, but you have to cross-correlate all possible pairs of the n vectors, so it can get ugly computationally as n gets large. $\endgroup$ – Jim Clay Feb 27 '12 at 20:20
  • $\begingroup$ @JimClay I am somewhat suspicious of the correlation test here ... cross-correlation gives us a score of how correlated two vectors are, but says nothing about how dependent they might be? (Example, you get a correlation score of 0, but they might still be dependent). Isnt this true? If yes, they how can we use it as a sort of test? $\endgroup$ – Spacey Feb 29 '12 at 1:14
  • $\begingroup$ @Mohammad Yes, there can be zero correlation even if the signals are dependent- particularly since we are only looking at a finite time window. That is why I have said on other threads that non-correlation is indicative of independence, but not proof. $\endgroup$ – Jim Clay Feb 29 '12 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.