How to use Robust PCA for image inpainting?

Question

I want to use Robust PCA for image inpainting, i.e. to estimate anomalous (bad) pixels. I assume that the bad pixels are known (mask).

I know how to calculate the Robust PCA using Alternating Direction Method of Multipliers but if somebody has already an implementation in Matlab I would not say no to sharing it. ;) But this is not my question.

How can I use Robust PCA for image inpainting, i.e. estimating the bad pixels? With Robust PCA I have X = L + S where X is my original image matrix for some training images, L is a low rank matrix and S is a sparse matrix.

Answer 1

As far as I can remember, one of the main interests of using R-PCA for image inpainting is that a pixel accurate defective pixel masks is not necessary. Intuitively, the main idea is that the observed image has low rank, but that the defects (given by the mask);

1. disturb this low-rank property
2. form a sparse distribution.

Thus, image inpainting using R-PCA is given in this context by:

$X = L + S,$

where $L$ is the restored (inpainted) image and $S$ is the defective pixel mask.

This approach was implemented for example for background subtaction (motion detection task): the authors perform a low rank reconstruction of some shopping mall hallway. The moving pedestrians end up in the sparse component (=> mask) and the background is given by the low rank part. The backgournd does not have any "holes" in place of the pedestrians since a hole breaks the low rank constraint.

For more insight about this approach, see Yi Ma's low rank page.