How can I compare two pictures from the camera and tell if there are enough differences to detect movement?

Question

I want to use my phone as a CCTV system to control a room in my house, and display an alert when something is moving.

For now, what I have succeeded in doing is to grab a picture every minute and upload them via a PHP script to my server. Now, I would like to compare the current picture and the picture 1 minute ago and detect if someone entered the room. So basically, I would need to compare differences of pixels on the picture (but also taking in account that a cloud may just say hello and changed brightness during one minute)

Does anyone have a clue on how to achieve that or some doc to read?

Answer 1

It seems to me what you are looking for is background subtraction technique. With noisy images and changing lighting conditions it could be nontrivial. The current state of the art technique for this is the low-rank matrix representation, but it require not two but many (~dozen) images. Some heavy duty math follow: Each image considered as a vector of pixels, vectors combined into matrix and this matrix is decomposed into low-rank matrix and remnant. Low-rank matrix columns are backgrounds and remnant is moving objects. There are some open-sourced implementations, but only for factorization itself, not complete image pipeline IIRC

Here is a paper and code for matrix factorization http://www.ece.rice.edu/~aew2/sparcs.html

Here is overview from CS blog and link to other code:

https://sites.google.com/site/igorcarron2/matrixfactorizations

http://nuit-blanche.blogspot.com/search/label/MF

Survey of some other techniques: http://www.vis.uni-stuttgart.de/uploads/tx_vispublications/Brutzer2011-2.pdf

Answer 2

Remember: There are shadows (not just brightness).

I don't really like the idea from James Webster, since it is basically scaling down the resolution in each dimension and comparing the scaled down images (also I don't like thresholds - they are to arbitrary, you have to test and tweak them until you get a good value and the next day these values could be obsolete because of different weather or some other environmental effects)

But to be fair, I also don't have a really good solution. My first idea was to diff each image to a reference image of the empty room and run an edge detection on the diff image. But this would also detect shadows. But I guess you can't differ between shadows and other objects...at least I don't know how. But maybe you can compare the result after the Edge detection between 2 frames, since shadows are mostly moving slowly (still there will be problems when cars move by or when a cloud moves by)

Answer 3

The "matrix factorization" technique will NOT help you do your job! The paper referred by @mirror2image is about the background subtraction but NOT based on "matrix factorization".

Using running video to detect moving objects (be it human or vehicles) is an active area of research.

As a basic principle the system estimates a typical static background by sampling over multiple pictures and take a difference of energy between incoming image to the background. If the energy is significant the pixel is classified as foreground. Such set of foreground tells you if there is an entry of the object in the system.

The best reference to your research paper (and also relatively simpler if you want to really implement) would be - W4 System find it here and see Picardi paper here as a more detailed survey for other techniques in the system.

There are many challenges that applies to the problem:

1. Presence of noise creates the issues of major ambiguity. The approach here is to apply efficient temporal filtering and considering variance of noise to make it immune to threshold.

2. Presence of shadow creates ambiguity of neither being a foreground nor. There are papers who model the color vs. intensity distinction to distinguish shadow vs. real foreground.

3. The background can be complex like waving trees or sea etc.

4. The background can have slow or sudden variation of lighting where earlier "learned" background is then adapted to the new one.

One of the most referred landmark paper is called Wall flower algorithm shows the best way to combine various such scenarios to produce robust moving object detection.

Answer 4

I don't know exact solution, but you should make some kind of hash of the image; a smaller data set extracted from the image, which is comparable better than the whole image.

I think, color histogram is a good choice for it.

If you split your image to areas and make separate histograms for these areas, you can determine the position/path of the intruder.

Answer 5

Taking difference twice i.e. difference of differences may help.. so if the double derivative of pixels is larger than a threshold in some regions, so you may term it as someone entered the room. Change in brightness will give approximately constant gradient throughout the image, but taking the Hessian or double derivative will give a good indication of movement or major changes in the room.