I am able to find out the velocities of pixels using the Lucas-Kanade method. Now I want to use it for tracking. I am using a Kalman filter taking $x$, $y$, $v_x$ and $v_y$ ($(x,y)$ is position of a particular object pixel and $(v_x,v_y)$ is velocity of a pixel) as state variables of the Kalman filter. Now the problem I am facing is how will I get to select the pixels that are representing the object to be tracked?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Tracking objects can be simple to complex depending on what type of background you have, whether the background is static or moving, whether the object is clearly distinguishable or can share similar properties with background there by blending in the background, and whether we have unique object at hand or we are tracking multiple objects which may cross the trajectories.
The good news is, the solution to all of these exists. Depending on the nature of task at hand, you may choose simple to complex solution.
What exactly we need?
As you implement any tracking algorithm, you have an estimate of where the new position of the object is likely be. Now you need to verify that indeed the object is at the same position or different. The difference is the error that you feedback to your model. For this you need to identify/segment the object in the vicinity where your projected value. In other words you need to detect that object with in a given window.
If the object's appearance or key behaviour doesn't change quite drastically you can capture feature vectors from original reference and compare the same around the vicinity where your target is expected to be. This is search of feature vector within a window based on some distance criteria.
So the question boils down to - what should be the feature vector and what should be the distance matrix?
There are many possible answers. Writing down some of them (from simple to complex) with remark on when they might work.
Color based features: This include identifying mean or dominant color, color histogram over a local window etc. If the object color is unique and doesn't blend with background then you can identify object with this manner. See [Ref 1] and [Ref 2] below
Texture based features: There is a whole range of features here. The famous Gabor filter banks, then there is co-occurrence matrix, you can take gradient of images or find edges and have edge histograms over local regions, or you can identify Local binary patterns near the regions. See [Ref 3]. Also, many approaches combines Color and Texture information for more effectiveness. See [Ref 4] and [Ref 5]
Shape based features: Shape based technique works when the object you are tracking has a particular shapes that distinguishes from other objects and from background. This includes many techniques. See [Ref 6], [Ref 7] and [Ref 8] below.
More complex features:
If the above features doesn't work out, specifically if the object is complex and the background as well -then we can use these features:
SIFT - The scale-invariant feature transform allows to compute key points which characterises the object and hence you can compare the same object from past frame against a new frame. See [Ref 9]
SURF, FAST, ORB: - If SIFT turns out to be blocked because of patent issue, and due to it's speed, then you can use it's alternatives which are SURF, FAST, ORB. See [Ref 10] for a good comparison.
HoG and Haar: Finally you can use HoG - which is a good generic object detection framework. [Ref 11] and Haar based classifiers when the problems are really complex. [Ref 12]
I am not prescribing any specific solution, and none of the above could be your direct solution - yet this will help you thinking through the problem at hand. You can pick whichever feature works for you irrespective of tracking methods that are used in those respective papers.
Go through the paper Good Features to Track thoroughly and it will give you much better perspective on the same.
What about distance criteria?
For each different feature typically different distance measure should make sense. But, distance matrix is usually dependent on your feature vector itself.
[Ref 3]: Texture Features for Browsing and Retrieval of Image Data B.S. Manjunathi and W.Y. Ma
[Ref 5]: Multi-Object Tracking Using Color, Texture and Motion Valtteri Takala and Matti Pietikainen
[Ref 9]: Object Tracking using SIFT Features and Mean Shift Huiyu Zhou, Yuan Yuan, and Chunmei Shi
answered Mar 20, 2018 at 12:37
$\begingroup$
$\endgroup$
2
If the only thing that moves is the object, you might threshold the magnitudes of the flow vectors. Magnitudes which are greater than some threshold simply represent the optical flow components of the moving object, and nothing else.
answered Jan 8, 2014 at 9:02
-
$\begingroup$ ok. it means this method of tracking will be suitable for the video scenes where background should be static $\endgroup$– Amit_DSPJan 8, 2014 at 9:06
-
$\begingroup$ Yes, as you are only using motion information, and no other means of object detection. If you incorporate some texture knowledge (such as detection by SIFT), you might then use it to narrow down your area of tracking to the object boundaries only. Also check out background subtraction. OpenCV has many methods. There are other more complex ways to do it as well. $\endgroup$ Jan 8, 2014 at 9:08