# Particle filter Tracking

I am tracking a car using particle filter by making a rectangle around it, and I am using the state vector $[x, y, u, v, a, h]$, where:

• $x$, $y$ is the position of the body
• $u$ and $v$ are velocity in $x$ and $y$ direction
• $h = \mathit{height}$, and $a = \mathit{width/height}$ , height and width of the rectangle

My state equations for:

$$x = x' + uT$$ $$y=y'+vT$$

$u$ and $v$ remain same, and I am adding some random noise to it.

But I am not able to figure out, how should I change my $a$ and $h$ value, what should be the state equations for that. I am using histogram matching as the likelihood function.

• Do $a$ and $h$ change as the car moves through the view? I suppose both might. Perhaps just model their changes with minor noise perturbations? Or, if you know that moving in a positive x direction (possibly away from the camera) makes $h$ go down, then use that information.
– Peter K.
Jun 18, 2014 at 14:19
• Yes both a and h has to change with car , as it might move away from the camera or near to the camera. Is there better and deterministic way of changing a and h? Jun 19, 2014 at 6:20

The easiest way to do this is to keep track of the parameters a and h from detection. How did you pick out a car from the scene? a simplified method would be: background subtraction, followed by clustering, data association, find centroid of clusters => x,y. Also store the bounding box of the cluster, easiest way to keep an accurate a,h.

But I'm guessing you want to start with initial conditions a0,h0 and based on x,y,u,v update a,h. The problem is that a,h cannot be completely known using only x,y,u,v,T. You also need information about the imaging system or the environment.

Assuming we use a Cartesian coordinate system x=horizontal, and y=vertical

there are a few different ways to view this problem. If you are tracking cars from a bird's eye view (think google maps for instance) then for all cases

a = a'

h = h'

realistically there can be slight variations. for instance if the car is facing this way __ vs this way | the bounding box would change, but given a set of x,y,u,v we have no way of knowing the cars orientation. Concretely, if the car is headed in the same direction (red and green) the bounding box shouldn't change. If the car changes heading (blue car) the bounding box changes drastically,

h = width' and width = h'

recall

a = w /h

so

h = a'*h'

a = width / h = h' / (a'*h')

unfortunately, we have no way of knowing with absolute certainty that the car changed directions. We can make up some kind though. Since cars are non holonomic vehicles ( a car can move forwards and backwards instantaneously but not left and right instantaneously) we can come up with some criteria to figure out a cars direction. For the red car we see |v| > |u| and |v'| > |u'|, or for the green car |u| > |v| and |u'| > |v'| but for the blue car |v| > |u| and |v| < |u| in this case we could guess that the car changed directions and the bounding box needs to be rotated using our second set of equations. However, if u~=v all bet's are off and we can't safely say which direction the car is pointing.

More complicated still is an image with perspective. For the following examples please note the origins in my picture are wrong, we should use the standard image convention where (0,0) is the upper left corner and (m,n) is the lower right corner where m=number of rows and n = number of columns. For this example I will use a image with a single vanishing point outside of the image. We can split this into a few cases

CASE 1: car is moves horizontally along x assuming the field of view is small we get

a = a'

h = h'

why? the height will only change as an object gets closer or further from us, since it is moving only along x, v=0 so our height shouldn't change. our width also shouldn't change (again the viewing angle must be small) because if we have a car traveling at velocity u, all points on the car move the x direction with the same u. More concretely, if delta_x = (x(T) - x(T-1)), delta_x should be the same for all points on the car. If the front tire moves delta_x (orange in pic), so should the back tire(light blue in pic), otherwise the car isn't moving as one solid body. IF both points moved the same delta_x, they should still be in the same position relative to one another (purple line in pic) so that means width doesn't change. we already said h doesn't change if the car moves only along x, and not y, so a shouldn't change either

CASE 2: car moves vertically, along y this is a bit more difficult. As far as I know there is no way to determine this without performing some kind of offline calibration. There are a few issues, first is that size alone can't determine distance in an image. Think of an image with a person, and mountains in the background. The person looks much larger than the mountains, but we know this isn't true, however, a computer would have more difficulty figuring this out. still using the single vanishing point outside of the image. We need to know something about the image, lets say we know the street is 6 meters wide (call this Xstreet). Now we measure the street at various y locations. at the top of the image Xstreet0 = 100 pixels. At the bottom Xstreet1 = 180 pixels. Lets also say that Xstreet0 and Xstreet1 are 100 pixels apart vertically. Even though the width of the street appears to be changing we know in reality the street is really 6m at both places. So we come up with some kind of equation to mirror our perspective. Remember this equation

y = Mx + b

lets use that, so to find our slope (we aren't really interested in b), which models our perspective

M = rise / run = (y1,y0) / (x1 - x0) = 100 / 80 = 1.25

that means for every 1 pixel we move downwards in the y direction our x pixel gets 1.25 times bigger! With this knowledge we can now do some modeling

h ~= h' (this is really difficult to determine, it gets into perspective and turns very messy, since we said there is only 1 vanishing point outside of the image we can say this doens't change much)

width = ((v*T)/M) + width'

looking closer we see that

v*T = delta_y

delta_y = M * delta_x ==> delta_x = delta_y / M ==> delta_x = (v*T)/M

so the term

(v*T)/M

gives us the change in X (delta_x), we simply add this to the old X to get our new X. but we aren't done yet a = width/height not simply width! so lets keep going

h = h' + delta_h

width = width' + delta_width

a = width / height = (width' + delta_width) / (h' + delta_h')

recall h ~= h' so delta_h = 0

a = (width' + delta_width) / h' = width'/ h' + delta_width / h' = a' + delta_width / h'

well we just calculated delta_width == delta_x, we know h' so we have our final equation

a = a' + (v*T)/(M *h')

where M was our slope (perspective compensation) we calculated offline.

CASE 3: a movement in x,y

is really just a movement in y, we have already said any change in x doesn't change the parameters at all. only a change in y

h = h'

a = a' + (v*T)/(M *h')

This problem gets a little more complicated when the vanishing line is actually in the scene. In this case, we have to keep track of if we are below or above the vanishing point and signs become very important.

This method also assumes that the objects we are tracking are in a constant z-plane. If our scene has a 3d axis, in our example, the street would be at one z value and cars travel in the x,y plane. We assume the cars don't levitate up and down, or our assumptions about y movements become invalid.

I mentioned it briefly, but this also won't work with wide FOV (field of view)cameras because our assumption that moving along x doesn't change width or height becomes invalid. If this doesn't make sense, just watch some fisheye video clips, like this video around 0 minutes 50 seconds

conclusion Overall, my methods are highly approximate, and only work under certain conditions (assuming a camera doesn't move often, they are fairly reasonable restrictions). The absolute best way to keep an accurate a,h would be to save the bounding box from the original car classification steps. Hope this helps someone in the future

You should change these variables according to a relevant generative model. In that case, this corresponds to the size of the car. Assuming cars having constant sizes this model should incorporate forinstance the laws of perspective.

If you describe your setting more precisely I could be more explicit.