2
$\begingroup$

I have been learning about using the Singular Value Decomposition to find low rank approximations to matrices. I had an image which I converted to a matrix. I regarded each row of the matrix as a 'data point'.

I did two things:

  1. Found the mean of the data points then subtracted the mean from each data point and calculated the SVD of the result
  2. Directly calculated the SVD of the original data matrix

Then I calculated the low rank approximations of the results from 1. and 2. by replacing some of the singular values with zeros. Finally I converted the results back to images.

Both images looked quite similar. They both looked like how SVD-compressed images should look (according to textbooks).

From a principal component analysis point of view, not mean-centering the data before finding the low-rank approximation shouldn't work well. It is equivalent to calculating the principal components from the eigenvectors of $\frac{1}{n-1}X^TX$ (where $X$ is the data matrix) instead of the covariance matrix of $X$. So why did it appear like mean-centering didn't matter? Or does it depend on the type of images?

$\endgroup$
0
$\begingroup$

I will try to illustrate why it is important to remove the Mean from data when doing the PCA (SVD is the tool to basically do the PCA approach to dimensionality reduction -> Compression).

If the data isn't centered you will have one (Most probably the most dominant) component (Direction) which will point to the center of the data.
Since all directions must be orthogonal, it means the next directions, which should be correlated to the data itself will have harsh constraint.

You want the directions to be chosen according to the data layout and not according to its position in space.

$\endgroup$
  • $\begingroup$ Yes, I understand that, which is why I am quite confused. If it's important to remove the Mean when doing PCA, how come when I didn't remove the mean before applying SVD, the compressed image look as good as removing the mean then applying SVD? Intuitively, not removing the mean should give noticeably worse result, no? $\endgroup$ – Noppawee Apichonpongpan Jul 23 '18 at 8:33
  • $\begingroup$ You need ot share what you're exactly doing in order to asses what's going on. When you deal with compression you need to ask given a set quality how many data needs to be retain or how high the quality given an amount of data. $\endgroup$ – Royi Jul 23 '18 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.