3
$\begingroup$

I have a Raspberry Pi NoIR Camera. Which I want to take pictures with at daytime in a well lit room.

Unfortunately the colors on the pictures do not look anything like you would expect from a normal camera. I guess the reason is:

(NoIR = No Infrared.) This means that pictures you take by daylight will look decidedly curious, but it gives you the ability to see in the dark with infrared lighting.

This post states that you can only overcome this issue during daylight by using an additional infrared filter or by using a "normal" camera.

So the pictures seem to lack any green values, plus some of the black values appear red/purple.

enter image description here

I could apply filters to the images programmatically or using gimp and I was wondering how would one tackle such a task? Could this be solved with some simple color correction algorithms? Is it even possible to revert this effect without using a additional "hardware"-infra red filter?

Thanks in advance

$\endgroup$

3 Answers 3

3
$\begingroup$

Unfortunately, no practically known signal processing method could provide what you wanted yet. Instead, you must use a hardware optical IR filter that would block the incoming IR waves, before they reach into your sensor.

An optical image is an array of intensity values captured by the sensor pixels which are illuminated by the incoming electromagnetic wave during the exposure time. During the capturing process, the static image sampling is performed on the 2D sensor surface, unlike an audio sampling which captures the time based variations of the incoming acoustic pressure wave.

Now the static-image sampling does not capture the time variations of the incoming electromagnetic wave. So those array of pixels represent only the spatial distribution of the illumination intensity $I(x,y)$ accross the sensor surface, over which an image was formed by a focusing lens system. Therefore, time-based property of light's wavelength is not represented in the samples, and cannot be processed later for modification.

Note that the camera system also samples the wave along temporal (time) dimension, the motion-picture; however it's typically about a few hundred frames per second at max. Given the required bandwidth of the visible spectrum; about $10^{14}$ Hz (visible light in [400-700] nm, plus IR spectrum to at least 1000 nm for near IR) this would require a sampling rate on the order of $10^{14}$ Hz. Such high data-rate is not practical for commercial applications of typical imaging devices.

So, for your Sony NoIR cam, the temporal-spectral information of the incoming electromagnetic wave is not captured, not represented by the image pixels, and therefore cannot be processes by any further means.

You should therefore physically place an IR blocking filter in front of the camera sensor, or the lens. Old GSM phones with built-in cameras, may incorporate an IR filter..?

$\endgroup$
0
$\begingroup$

It is possible to retrain an CNN designed to color correct underwater images but instead trained with IR-cut/No IR-cut pairs of images. Ideally these pairs should come from the same camera and of the identical scene where IR-cut is activated and deactivated, or by manually introducing an IR-cut filter between images. Note the importance of subject matter being more or less identical between each image pair.

Here is one such CNN for underwater color correction. I'm not sure if this represents the state-of-the-art but it has a training routine and is well documented.

Underwater GAN (UGAN)

$\endgroup$
0
$\begingroup$

Let $\hat q_r$, $\hat q_g$, $\hat q_g$ be the estimated number of photons collected at each of your red, green and blue pixels.

Approximately, with an IR cut filter, $\hat q_r = k_r q_r$, $\hat q_g = k_g q_g$, $\hat q_b = k_b q_b$, where $k_x$ is a pixel's gain in its intended color, and $q_x$ is the number of photons hitting the pixel. Not only three equations and three unknowns, but three easy equations in three unknowns.

Take out the cut filter, and you get $$\hat q_r = k_r q_r + k_{ir} q_i \\ \hat q_g = k_g q_g + k_{ig} q_i \\ \hat q_b = k_b q_b + k_{ib} q_i$$

Now you have three equations, but even if you don't want to know it, $q_i$ is a fourth unknown. Because of material variations and different illumination levels, you can't know what $q_i$ is for each pixel, so you can't really ever solve for the three unknowns that you do want.

Your very best bet is a camera with a cut filter. Second is a cut filter taped over your camera. A distant third is to try to correct for IR after the fact, but it'll be hard to get there from here.

(as a fun science experiment, compare the colors you see with the same scene illuminated only by diffuse sunlight, and the scene with an LED lamp shining on parts of the scene -- an LED lamp + no sunlight should give you fairly natural colors due to the absence of near IR. Sunlight (or incandescent light) should give you lots of near IR, and lots of color distortion).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.