I am trying to obtain an intuitive understanding of Eigen Values of covariance matrix and have used a few layman terms because I fully do not understand the concept yet.
The following is the code snippet written in Octave.
clear;clc; #p,q,r,s,t,u,v and w are features. #Each of them have been measured 10 times # p,q and r move in one exact direction [ascending] p = [1,2,3,4,5,6,7,8,9,10]; q = (p * 0.5) + 4; r = p .* q; # s moves in descending direction s = -r; t = s/2; # t should show up flat in all scatter plots u = [1,1,1,1,1,1,1,1,1,1]; # v and w are random v = rand(1,10); w = rand(1,10); # Create a snapshot matrix # Columns are features and rows their observations X = [p',t',q',r',w', s',v',u']; # Compute the covariance matrix R = cov(X); # Compute Eigen Vectors and corresponding Eigen Values [EigVec, EigVal] = eig(R); octave>>diag(EigVal) -1.1581e-014 0.0000e+000 2.7459e-031 1.4409e-018 5.3430e-002 8.4624e-002 2.0351e-001 1.9057e+003
My understanding is that the Eigen-Vectors with significant Eigen-Values of covariance matrix represent significant underlying directional forces in the datasets.
In my example, I think there are three directional forces:
p,q,r => Ascending direction
s,t => Descending direction
u => Horizontal direction
v,w => Random and must not show up as a definitive underlying pattern
I am therefore expecting 3 significant Eigen Values in the output, but there seems to be only two [1.9057e+003 and 2.0351e-001 - possibly for ascending and descending directional forces].
My question hence is why is the horizontal directional force not showing up in the Eigen value output?