How Does Mean Centering Affect the Result of Using SVD to Compress Images?

Question

I have been learning about using the Singular Value Decomposition to find low rank approximations to matrices. I had an image which I converted to a matrix. I regarded each row of the matrix as a 'data point'.

I did two things:

1. Found the mean of the data points then subtracted the mean from each data point and calculated the SVD of the result
2. Directly calculated the SVD of the original data matrix

Then I calculated the low rank approximations of the results from 1. and 2. by replacing some of the singular values with zeros. Finally I converted the results back to images.

Both images looked quite similar. They both looked like how SVD-compressed images should look (according to textbooks).

From a principal component analysis point of view, not mean-centering the data before finding the low-rank approximation shouldn't work well. It is equivalent to calculating the principal components from the eigenvectors of $\frac{1}{n-1}X^TX$ (where $X$ is the data matrix) instead of the covariance matrix of $X$. So why did it appear like mean-centering didn't matter? Or does it depend on the type of images?

Answer 1

I will try to illustrate why it is important to remove the Mean from data when doing the PCA (SVD is the tool to basically do the PCA approach to dimensionality reduction -> Compression).

If the data isn't centered you will have one (Most probably the most dominant) component (Direction) which will point to the center of the data.
Since all directions must be orthogonal, it means the next directions, which should be correlated to the data itself will have harsh constraint.

You want the directions to be chosen according to the data layout and not according to its position in space.