Transforming RGB images including NIR to LAB

Question

I'm relatively new to image processing, so I hope I don't ask trivial questions.

I have some images that I want to use in a machine learning context. The images have four color channels: RGB and NIR (near infrared). After some research, it seems like the LAB color space is better suited for machine learning in many cases than the RGB one.

First question would be: is that correct, or did I just fall for single people claiming something?

The second, more important question: Is there a way to properly include NIR in the LAB color space? Or would it be better to keep it as separate information?

And lastly, another question that came up: Can I even transform RGB to LAB properly? What information would I need (wavelengths of red, green and blue; some information on the whitepoint, ...)? Is there a different color space better suited?

If relevant: I am focusing on plant images and tasks appearing there, e.g. plant/background segmentation, fruit detection, leave counting,...

Answer 1

Let's start with the source of your RGB + near IR images. To break the process down:

1. Some camera must have taken those images.
2. The camera sensor is sensitive to light within two given wavelengths. The technical term for this is spectral response.
3. Alternating elements (Bayer pattern) in a sensor are etched with filters. Each filter type passes certain wavelengths while supressess the others.
4. The alternating sensor elements accumulate the filtered light from different wavelengths. This creates our very first raw signal. This raw space is then mapped onto a color space, most commonly the sRGB space.
5. Filter properties are precisely measured by the manufacturers. They use those properties (and assume a D65 white point, unless manually chosen to something else) to convert the captured raw space into sRGB which gets passed down to you (usually with a little compression and anti-aliasing filtering).

Answering your final question first, with the sRGB assumption in place, you are now armed with one very important source of information - your white point is D65 unless explicitly stated elsewhere. You may now safely proceed to convert your sRGB image into LAB. MATLAB has a handy function for the same.

To answer your questions:

First question would be: is that correct, or did I just fall for single people claiming something?

A: People say a lot of things - largely from their personal experience. You must create your own. I personally exect LAB to help. You can read more about that on my Quora answer here.

Is there a way to properly include NIR in the LAB color space? Or would it be better to keep it as separate information?

All traditional color spaces are designed to fall within the locus of human perception. Since near IR is out of bounds for our perception, it's usually modelled as a standalone entity (seperate channel). However, many imaging devices have a near IR spectral response below 750 nm. If that's the case, I would expect statistical correlation between your G or R channels with near IR. A simple PCA would answer that question.

Answer 2

My answer focuses on the machine learning part instead of the original question concerning LAB color space.

From a Machine Learning perspective (and more specifically Deep Learning with semantic segmentation networks for your application) taking as input all of the four bands seems optimal unless you can assert that one (or more) channel(s) is (are) not relevant for the segmentation task.

The network is expected to build its own representation of the image and experiments show that firsts layers learn transformations such as color space changers/selectors, blob and edge detectors.

For plant detection green and NIR components are the most discriminant channels (see EGI and NDVI from literature) but others could participate (for ground segmentation for instance).

I suggest training a machine learning model with different inputs (RGB only, B&W, RGB+NIR, EGI, LAB) and then to compare performances between those different approches. You’ll be able to better evaluate the influence of each spectral band in the result.