0
$\begingroup$

Context: I'm trying to improve a pose estimation model so that it works better when my camera is in Infrared mode. Unfortunately I only have RGB images to train on.

I realize that you can't convert RGB to IR directly, but my hypothesis is that converting the RGB images to look more like IR, and then training on a dataset of combined RGB and IR images, will lead to better performance.

Are there any libraries that have tried to implement a function like this? I'm essentially looking for a function that something like this ("IR effect") - http://funny.pho.to/infrared/

Improve this question
$\endgroup$
3
  • $\begingroup$ I have the same problem, and I want to convert RGB images to IR images (the ones that look like grayscale). Did you find any solution? I would appreciate if you share your search result if you found a way to do that. Thx $\endgroup$
    – user57711
    Jun 3, 2021 at 10:15
  • $\begingroup$ @user57711 Welcome to SE.SP! If the selected answer (the one with the check mark) doesn't work for you, please ask a new question. Please try to say what doesn't work about this question's answer so we can help identify the issue. $\endgroup$
    – Peter K. ♦
    Jun 3, 2021 at 13:01
  • $\begingroup$ @user57711 SE.SP is not a discussion forum. Please do not post an "answer" that does not answer the question. As I said before, please ask a new question. $\endgroup$
    – Peter K. ♦
    Jun 3, 2021 at 13:45

3 Answers 3

Reset to default
3
$\begingroup$

Without very much additional info, I'd presume the red channel of your camera has the highest correlation with the infrared spectrum.

Since you only have RGB and no knowledge of how that was calculated from the color sensor pixels: Take the R channel, it's as good as it gets.

The "IR effect" is for artistic purposes only. I'd doubt it has anything to do with what you see in an IR picture.

Notice that I also think you haven't tried to properly understand the physics of your problem: While I'd presume that a visible light photograph is highly correlated with a near-IR photo, that's very likely not the case for mid- and long-wavelength IR and certainly not for far infrared.

So, define your requirements better. And: ML really depends on having data. A lot of it. you might start with wrong data, but as DNN really don't lend themselves to understanding of the weights, it's very questionable that anything that you do with "emulated" data transfers to "real" IR data well.

Improve this answer
$\endgroup$
2
  • $\begingroup$ When the camera is in IR mode, the IR filter is mechanically removed and the images look black & white. So to clarify, I think the images I'm looking at are near-infrared and look pretty similar to black and white images. With that said, should I just convert to grayscale or are there better ways to convert an RGB image to something more like near-infrared? $\endgroup$
    – megashigger
    Feb 13, 2018 at 19:46
  • 1
    $\begingroup$ just drop everything but the R channel. $\endgroup$
    – Marcus Müller
    Feb 13, 2018 at 20:05
2
$\begingroup$

The real result of the image is a combination of three channels with a different weight. The IR mode of a camera just remove the IR filter(cutting over 2 um) letting the full spectrum of the light to reach the CMOS sensor. Silicon has high sensitivity around 500 nm (green light) so it is important to keep these information. I would suggest to increase the weight of red and reducing the blue one, before transforming the data into BW image.

Improve this answer
$\endgroup$
1
$\begingroup$

Near IR images are typically monochrome images from cameras where the IR filter has been removed, causing the inherent sensitivity of IR of camera sensors to dominate, and/or an extra filter that attenuates visible light.

Typical characteristics of IR images is that foliage become very bright, sky and water is dark, eyes become «creepy», skin artifacts are reduced. I think it would be hard to transform rgb images to simulate IR to the degree that you can train something to detect uniquely IR features on it.

http://www.wrotniak.com/photo/quest/_img/e510-ir.x.jpg

https://upload.wikimedia.org/wikipedia/commons/4/4d/UV_Vis_IR_Portrait.jpg

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.