Linear correlation of two vectors?

Question

We had a lecture on digital watermark detection using linear correlation. Here it was explained that the linear correlation of two vectors is equal to their scalar product divided by the dimension. So, the larger the scalar product, the larger the linear correlation.

This is not true, is it?

Simple example:

u = [1,2,3,4]
v1 = [10,10,10,10]
v2 = [0, 1,2 3]

The scalar product u*v1 would be way bigger than u*v2, but obviously, there is a perfect correlation between u and v2 ? Can anyone guess how the statement was to be understood?

Thanks!

Answer 1

You can use corrcoef to compute the Correlation coefficients as a measure of linear dependence of the random vectors (i.e you transpose your row vectors, the variables are column vectors, and the rows are observations). With the vectors

u = [1, 2, 3, 4];
v1 = [10, 10, 10, 10];
v2 = [0, 1, 2, 3];

You have

>> R = corrcoef([u', v1', v2'])

R =

     1   NaN     1
   NaN   NaN   NaN
     1   NaN     1

There is a perfect match between u and v2 as v2' = u' - 1 showing that the two variables u and v2 are directly correlated. In the results above R is $$ \mathbf R = \begin{pmatrix} \frac{\operatorname{cov}(\mathbf u, \mathbf u)}{\sigma_{\mathbf u}^2}& \frac{\operatorname{cov}(\mathbf u, \mathbf v_1)}{\sigma_{\mathbf u}\sigma_{\mathbf v_1}} & \frac{\operatorname{cov}(\mathbf u, \mathbf v_2)}{\sigma_{\mathbf u}\sigma_{\mathbf v_2}}\\ \frac{\operatorname{cov}(\mathbf v_1, \mathbf u)}{\sigma_{\mathbf v_1}\sigma_{\mathbf u}}& \frac{\operatorname{cov}(\mathbf v_1, \mathbf v_1)}{\sigma_{\mathbf v_1}^2} & \frac{\operatorname{cov}(\mathbf v_1, \mathbf v_2)}{\sigma_{\mathbf v_1}\sigma_{\mathbf v_2}}\\ \frac{\operatorname{cov}(\mathbf v_2, \mathbf u)}{\sigma_{\mathbf v_2}\sigma_{\mathbf u}}& \frac{\operatorname{cov}(\mathbf v_2, \mathbf v_1)}{\sigma_{\mathbf v_2}\sigma_{\mathbf v_1}} & \frac{\operatorname{cov}(\mathbf v_2, \mathbf v_2)}{\sigma_{\mathbf v_2}^2} \end{pmatrix} $$

The NaN results between (v1', u1'), (u', v1'), (v1', v1'), (v1', v2') and (v2', v1') is because v1' is constant, see MATLAB documentation on corrcoef, more specifically because

$$\begin{cases} \operatorname{cov}(\mathbf v_1, \mathbf u) = \operatorname{cov}(\mathbf u, \mathbf v_1) = 0\\ \operatorname{cov}(\mathbf v_1, \mathbf v_2) = \operatorname{cov}(\mathbf v_2, \mathbf v_1) = 0\\ \operatorname{cov}(\mathbf v_1, \mathbf v_1) = 0 \end{cases}\quad\text{and}\quad\sigma_{\mathbf v_1} = 0 $$

In the end, each element of the matrix $\mathbf R$ is the covariance of two vectors divided by the product of their standard deviations.

EDIT:

Now let's say v1 isn't a constant vector and you have the three vectors as

u = [1, 2, 3, 4];
v1 = [11, 12, 11, 12];
v2 = [0, 1, 2, 3];

You then get

R =

    1.0000    0.4472    1.0000
    0.4472    1.0000    0.4472
    1.0000    0.4472    1.0000

Note that in the case here $\sigma_{\mathbf v_1} \neq 0$, $\operatorname{cov}(\mathbf v_1, \mathbf u) \neq 0$, and $\operatorname{cov}(\mathbf v_1, \mathbf v_2) \neq 0$