I am using Matlab function PCA (principal component analysis) to reduce the dimensionality of a data set with approximately 20 000 observations x 100 dimensions.

After having obtained the principal component coefficients of the data I recreated the input signal in the original coordinate system using the transformation matrix from the PCA function. This yielded a very large residual when comparing with the input signal. I have tried around with different data sets and sizes and it appears to be commonplace to have a large residual. I am not sure yet whether it is due to round-off errors or high SNR in the input data. The dimensionality reduction could of course still be useful, but is this something that one should be cautious about when performing principal component analysis? Or is there another metric that is better to assess the performance of the principal component analysis?

  • $\begingroup$ How many principal components do you keep? The residual error depends on the number of principal components you're reconstructing your data from. $\endgroup$
    – roygbiv
    Sep 26 '16 at 15:22
  • $\begingroup$ The error was that I didn't know that Matlab also centers the input data around zero prior to applying the PCA transformation. Now that I have realized that it works as it was intended. $\endgroup$
    – hamo
    Oct 3 '16 at 11:36

PCA dimensionality reduction seeks for a linear transformation, mapping the data to a lower dimensional space. The linearity comes from the mapping, which is simply a rotation (and translation if you take into account the de-meaning). In the end, it is nothing but a projection of the data.

So if your low dimensional embedding is not reconstructing the data truthfully, then this means that your data is not suitable for such linear mapping, i.e. not linearly separable. You should look into other techniques such as linear discriminant analysis (canonical variates), Kernel-PCA, which applies kernel trick to achieve non-linearity or manifold learning.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.