# Interpretation of Eigen Values of covariance matrix

I am trying to obtain an intuitive understanding of Eigen Values of covariance matrix and have used a few layman terms because I fully do not understand the concept yet.

The following is the code snippet written in Octave.

clear;clc;

#p,q,r,s,t,u,v and w are features.
#Each of them have been measured 10 times
# p,q and r move in one exact direction [ascending]
p = [1,2,3,4,5,6,7,8,9,10];
q = (p * 0.5) + 4;
r = p .* q;

# s moves in descending direction
s = -r;
t = s/2;

# t should show up flat in all scatter plots
u = [1,1,1,1,1,1,1,1,1,1];

# v and w are random
v = rand(1,10);
w = rand(1,10);

# Create a snapshot matrix
# Columns are features and rows their observations
X = [p',t',q',r',w', s',v',u'];

# Compute the covariance matrix
R = cov(X);

# Compute Eigen Vectors and corresponding Eigen Values
[EigVec, EigVal] = eig(R);

octave>>diag(EigVal)
-1.1581e-014
0.0000e+000
2.7459e-031
1.4409e-018
5.3430e-002
8.4624e-002
2.0351e-001
1.9057e+003


My understanding is that the Eigen-Vectors with significant Eigen-Values of covariance matrix represent significant underlying directional forces in the datasets.

In my example, I think there are three directional forces:

p,q,r => Ascending direction

s,t => Descending direction

u => Horizontal direction

v,w => Random and must not show up as a definitive underlying pattern

I am therefore expecting 3 significant Eigen Values in the output, but there seems to be only two [1.9057e+003 and 2.0351e-001 - possibly for ascending and descending directional forces].

My question hence is why is the horizontal directional force not showing up in the Eigen value output?

• The short answer is that your vectors are not orthogonal and lay primarily in a sub space that only requires 2 vectors. – Stanley Pawlukiewicz Mar 17 '18 at 20:15
• When you say vectors are not orthogonal, you are referring to the data p,q,r..w..correct? – Raj Mar 18 '18 at 3:04
• look at the SVD of X – Stanley Pawlukiewicz Mar 18 '18 at 3:22