Context: I'm trying to improve a pose estimation model so that it works better when my camera is in Infrared mode. Unfortunately I only have RGB images to train on.

I realize that you can't convert RGB to IR directly, but my hypothesis is that converting the RGB images to look more like IR, and then training on a dataset of combined RGB and IR images, will lead to better performance.

Are there any libraries that have tried to implement a function like this? I'm essentially looking for a function that something like this ("IR effect") - http://funny.pho.to/infrared/


Without very much additional info, I'd presume the red channel of your camera has the highest correlation with the infrared spectrum.

Since you only have RGB and no knowledge of how that was calculated from the color sensor pixels: Take the R channel, it's as good as it gets.

The "IR effect" is for artistic purposes only. I'd doubt it has anything to do with what you see in an IR picture.

Notice that I also think you haven't tried to properly understand the physics of your problem: While I'd presume that a visible light photograph is highly correlated with a near-IR photo, that's very likely not the case for mid- and long-wavelength IR and certainly not for far infrared.

So, define your requirements better. And: ML really depends on having data. A lot of it. you might start with wrong data, but as DNN really don't lend themselves to understanding of the weights, it's very questionable that anything that you do with "emulated" data transfers to "real" IR data well.

  • $\begingroup$ When the camera is in IR mode, the IR filter is mechanically removed and the images look black & white. So to clarify, I think the images I'm looking at are near-infrared and look pretty similar to black and white images. With that said, should I just convert to grayscale or are there better ways to convert an RGB image to something more like near-infrared? $\endgroup$ – megashigger Feb 13 '18 at 19:46
  • 1
    $\begingroup$ just drop everything but the R channel. $\endgroup$ – Marcus Müller Feb 13 '18 at 20:05

The real result of the image is a combination of three channels with a different weight. The IR mode of a camera just remove the IR filter(cutting over 2 um) letting the full spectrum of the light to reach the CMOS sensor. Silicon has high sensitivity around 500 nm (green light) so it is important to keep these information. I would suggest to increase the weight of red and reducing the blue one, before transforming the data into BW image.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.