7
$\begingroup$

I am working on developing an autonomous control system for toy IR remote-controlled helicopters (like the Syma s107, for example) using a webcam and an Arduino with some IR LEDs. I am using C++ with OpenCV on a laptop running at about 2Ghz and I need to keep the system as close to real time as possible.

I am unsure of a way to effectively determine the rotation of the helicopter in the 2D image (more precisely its yaw with respect to the camera image plane). This information is important to being able to stabilize the helicopter in the screen because otherwise it floats out of view.

I have considered, and at least partially tried to determine the location of the body and the tail and use some tricks of geometry and perspective to get an estimation of shape but accurately finding the tail and body with low enough noise proved difficult. I have also considered using colored markers of some sort on certain spots of the helicopter but I'm not sure how to get a more accurate estimation than approximately left and approximately right.

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Can you post some (preferably not big) sample pictures and highlight the problems with them? $\endgroup$
    – h3nr1x
    Jan 9, 2013 at 5:22
  • $\begingroup$ @mtrw I don't know. This is more of a image recognition problem. It is not, however, a bit of image recognition that I know anything about. $\endgroup$
    – dmckee --- ex-moderator kitten
    Jan 9, 2013 at 5:34
  • $\begingroup$ OK. I take it back. Their most active tag is [image-processing]. This might be well suited to dsp.se. $\endgroup$
    – dmckee --- ex-moderator kitten
    Jan 9, 2013 at 5:36

3 Answers 3

Reset to default
4
$\begingroup$

For yaw it's sufficent to have three distinct data points fixed to the helicopter in a plane parallel to the rotor rotation: different colored leds for example. You could put them on three thin radial arms fixed to the landing gear. Maybe colored reflective materials would work, too, but you'd have to compensate for environmental light changes.

Then getting the yaw angle is simple. Suppose the lights are red, green, and blue 120 degrees apart and ccw when viewed from the top.

enter image description here

Then in the image you will see these lights in various x-coordinate orders. As the helicopter rotates ccw fromt the position shown in the diagram:

G R B   // green and blue in front; red in back
R G B   // green in front; red and blue in back
R B G   // red and gree in front; blue in back
B R G   // etc...
B G R
G B R

(Of course at the borders of these regions two lights will merge to a single point. These can be handled as special cases.)

So with 3 lights you have split all posible yaws into 6 x 60 degree segments.

You can further get down to 30 degree resolution by just comparing the left and right "gap" widths. If pointing directly away from the camera is azimuth theta zero degrees and the red light is on the nose (as in the picture), then the algoritm is:

Xr = -sin(theta)
Xg = -sin(theta + 60)
Xb = -sin(theta + 120)
if (Xg < Xr && Xr <= Xb) {
  // Green and blue are in front of red
  dxLeft = Xr - Xg
  dxRight = Xb - Xr
  yaw = (dxLeft < dxRight) ? 15+-15 : -15+-15;
}
else if (Xr < Xg && Xg < Xb) {
  // Green is in front of red and blue
  dxLeft = Xg - Xr
  dxRight = Xb - Xg
  yaw = (dxLeft < dxRight) ? 30+-15 : 60+-15;
}
else ... four more cases

The X+-15 above mean X degrees plus or minus 15: a 30 degree segment. To get the exact offset, you can construct a preprocessed table that converts the ratio 

R = dxLeft < dxRight ? xLeft / xRight : xRight / xLeft

into an an exact azimuth offset between -15 and 15. This is pretty simple trigonometry to work out, and the nice thing is that it works independent of how far away the helicopter is, so long as it never turns upside down!

Another side benefit: the image distance between the outer two lights gives you inverse distance to the camera.

Improve this answer
$\endgroup$
2
  • $\begingroup$ You may also need take into account the location of the helicopter relative to the camera (at least in the X direction), as this will likely affect how the distances between the LEDs appear. But I think some clever trig work should allow you to compensate for it. $\endgroup$
    – Bill
    Jan 9, 2013 at 16:21
  • $\begingroup$ You'd need to know the focal properties of the camera optics and imaging plane to be certain, but as long as the helicopter is not extremely close, off-axis effects will be small. $\endgroup$
    – Gene
    Jan 9, 2013 at 16:29
1
$\begingroup$

The markers idea could work.

 ___ ___                ___  ___ 
  m | m                  m /  m
    |                     /   
    +                    +   
    |                   /   
    |                  /   
 --- ---           --- --- 


   \ /                   \ /        
    c                     c

Imagine putting markers on the far side of the helicopter and having them blocked on the near side. When the helicopter rotates, the marker on the approaching side will be exposed to the camera.

Improve this answer
$\endgroup$
1
$\begingroup$

Full location and orientation of the copter can be calculated with: 4 known points on a plane + calibrated camera, or with 6 known points (certain 3D configuration will not work, for example the points must not be on a same plane). See this question for 6 point algorithm, for 4 point algorithm I have found MSc thesis by Liljequist - Planes, Homographies and Augmented Reality very useful, but I can not find it on-line. More terse paper which focuses on the calibration is written by Zhang - A Flexible New Technique for Camera Calibration.

Good book about the topic: Hartley, Zisserman - Multiple view geometry in computer vision.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.