I am trying to obtain an intuitive understanding of Eigen Values of covariance matrix and have used a few layman terms because I fully do not understand the concept yet.

The following is the code snippet written in Octave.


#p,q,r,s,t,u,v and w are features. 
#Each of them have been measured 10 times 
# p,q and r move in one exact direction [ascending]
p = [1,2,3,4,5,6,7,8,9,10];
q = (p * 0.5) + 4;
r = p .* q;

# s moves in descending direction
s = -r;
t = s/2;

# t should show up flat in all scatter plots
u = [1,1,1,1,1,1,1,1,1,1];

# v and w are random
v = rand(1,10);
w = rand(1,10);

# Create a snapshot matrix 
# Columns are features and rows their observations
X = [p',t',q',r',w', s',v',u'];

# Compute the covariance matrix
R = cov(X);

# Compute Eigen Vectors and corresponding Eigen Values
[EigVec, EigVal] = eig(R);


My understanding is that the Eigen-Vectors with significant Eigen-Values of covariance matrix represent significant underlying directional forces in the datasets.

In my example, I think there are three directional forces:

p,q,r => Ascending direction

s,t => Descending direction

u => Horizontal direction

v,w => Random and must not show up as a definitive underlying pattern

I am therefore expecting 3 significant Eigen Values in the output, but there seems to be only two [1.9057e+003 and 2.0351e-001 - possibly for ascending and descending directional forces].

My question hence is why is the horizontal directional force not showing up in the Eigen value output?

  • $\begingroup$ The short answer is that your vectors are not orthogonal and lay primarily in a sub space that only requires 2 vectors. $\endgroup$ – Stanley Pawlukiewicz Mar 17 '18 at 20:15
  • $\begingroup$ When you say vectors are not orthogonal, you are referring to the data p,q,r..w..correct? $\endgroup$ – Raj Mar 18 '18 at 3:04
  • $\begingroup$ look at the SVD of X $\endgroup$ – Stanley Pawlukiewicz Mar 18 '18 at 3:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.