PCA to reduce dimensionality to 99% variance

Question

I'm attempting to use PCA to reduce the dimensionality of a dataset I have. I want to explain 99% of the variance in the dataset, and I think I've been able to determine that, but I'm unsure what I have to do to my original dataset to reduce the dimensions. Here's what I've got so far (MATLAB code):

rng 'default'
M = 1000; % Number of observations
N = 500; % Number of variables observed
X = rand(M,N);

% Remove mean.
X = (X - repmat(mean(X),[size(X,1) 1]));

% Determine the eigenvector (V) and eigenvalues (D) of the covariance of
% the matrix X.
[V, D] = eig(cov(X));

% Calculate variance of the vectors (largest to smallest).
var_vec = flipud(diag(D));

% Calculate the cumulative percentage of the variances.
percentages = cumsum(var_vec) / sum(diag(D));

% Find the 99% variance
PC99 = find(percentages >= 0.99, 1);

PC99 tells me the that components 1 to 464 (out of 500) contain 99% of the variance, which I believe to be correct, but I'm now unsure on how to manipulate my original data in order to reduce the dimension from 1000x500 to 1000x464.

Thanks for any help.

Answer 1

If what you want is PC compression, a more useful form of PCA would be achieved via Singular Value Decomposition (which is, in many cases, more accurate and faster than an eigendecomposition).

Assuming that X is a "tall" matrix with dimensions [n, p], n>p; Then:

 [U, S, V] = svd(X);

eig_vals = diag(S.^2); %recall that eigenvalues of cov(X) are the singular values of X, squared.
percentages = cumsum(eig_vals)./sum(eig_vals); % cumulative variance. 

% Find the 99% variance
PC99 = find(percentages >= 0.99, 1);

X_reduced = U(:,PC99)*S(PC99,PC99)*V(PC99,PC99)'; % compressed version of X

If X is a wide matrix [n < p], you have to adjust the above expression a bit (all of V will be used, but only part of U, unlike here where V is truncated as well). We leave it as an exercise to the interested reader!

I will leave wiser people to comment on the judiciousness of PCA for compression. In many cases there are much more efficient schemes, such as wavelet decomposition followed by "thresholding" (zeroing out the coefficients that fall below a certain threshold). But since your question assumes that you know what you want, so does the answer.