9
$\begingroup$

I need to detect how fast a camera is panning (either horizontal/vertical) to give a warning to the operator to slow down.

The entire image is moving as a block, I don't need an actual direction (although H or V would be a bonus) and I only need an approximate magnitude - ie. trigger if more than 'N' pixels shift between frames.

Images are large and generally uniform low contrast scenes, I don't have any obvious highlights to track. I need to do it in realtime (60fps) and without using all of the CPU.

Niave solution is pick an RoI in the center, find edges, calculate similarity between pairs of frames, shift one of frames left/right/up/down by a pixel, repeat - find minima.

I wondered if there was a smarter solution?

|improve this question
$\endgroup$
3
$\begingroup$

Probably if you are looking for a simple method, it is to apply the standard Motion Estimation algorithms which are very much matured in MPEG class of compression codecs. They are easy to understand and i guess you will get a lot of ready to use codes. This algorithm produces motion vector on block by block basis - and then you can find the most prominent cluster and take the average motion vector direction and magnitude.

MPEG4 - has another key concept called "Global Motion compensation", a technique which makes attempts to first estimate and compensate camera motion and panning. The beauty is that such methods can be simpler or exhaustive depending on complexity. Here is one example paper and another paper for the same.

In general, camera panning and motion estimation is quite an established research domain. here is a reference: paper and another paper.

On this subject. You will find both rigor and accurate algorithm as well as simple and fast ones.

|improve this answer
$\endgroup$
  • $\begingroup$ If I can conveniently hook into an MPEG lib that would be good, I remember that GMC in Mpeg had criticisms. I thought it would be a common area because of camera stabilisation algorithms $\endgroup$ – Martin Beckett Mar 31 '12 at 14:34
  • $\begingroup$ You can definitely hook (or rather extract) MPEG algorithms. You can use FFMPEG as library and extract that - but might get tricky. Alternatively, you can read neat code of MSSG to extract. $\endgroup$ – Dipan Mehta Mar 31 '12 at 14:40
  • $\begingroup$ Regarding the criticism on GMC - it is more of it's over promise to dramatically reduce the bit rate and create object based encoding. However, it is not really that hard to estimate camera motion parameters. $\endgroup$ – Dipan Mehta Mar 31 '12 at 14:41
  • $\begingroup$ thanks, I will take a look at MSSG. I use ffmpeg but it's not an easy library to just pull things out of! $\endgroup$ – Martin Beckett Mar 31 '12 at 14:54
3
$\begingroup$

This might be a slow terrible solution, but you could do a FFT-based cross-correlation of subsequent frames and then find the peak to identify the offset between frames. Maybe only do it on a small subset of the image to save processor cycles.

It would not work with rotation or drastic scene changes from one frame to the next, and there are probably better methods. This is kind of a "I have a hammer so everything looks like a nail" solution. I guess this is just like your naive solution, except there's no need for edge detection and the FFT makes it a lot faster than explicitly shifting one pixel at a time.

This question is similar, and no one's suggesting anything other than cross-correlation, so maybe it's not so bad: Using MATLAB to calculate offset between successive images

|improve this answer
$\endgroup$
3
$\begingroup$

One way you could estimate the velocity and the direction would be to make a "local" flow estimation of e.g. four windows in the center of image. The Lucas–Kanade differential method assumes the displacement is approximately constant and it is therefore possible to solve as an equation.

So my step-by-step guide would be:

  1. Get a window of pixels in the center of the image, e.g. 20x20
  2. Calculate the gradients Ix and Iy.
  3. Split the gradient window into four pieces, e.g. 4x10x10.
  4. Solve the four linear least squares equations with next frame.
  5. Average the four velocity vectors.

This determines the direction and velocity, however you could use a weighted window to make it more robust. Look at the Lucas-Kanade method for its extensions.

|improve this answer
$\endgroup$
0
$\begingroup$

I think cross-correlation is a good approach to find the offset, but if you want to do it real fast then you could try to restrict it to only a single vertical and a single horizontal scanline (i.e. through the center of the image.) Computing the cross-correlation between the scanlines in both frames should give you an approximation of the horizontal and vertical offset.

|improve this answer
$\endgroup$
  • $\begingroup$ This might work, but if it's panning diagonally, it won't work well, even if it's just a little jitter up and down while panning sideways. I think a rectangular subregion in the center of the image would be better. $\endgroup$ – endolith Apr 4 '12 at 0:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.