0
$\begingroup$

I'm relatively new to image processing, so I hope I don't ask trivial questions.

I have some images that I want to use in a machine learning context. The images have four color channels: RGB and NIR (near infrared). After some research, it seems like the LAB color space is better suited for machine learning in many cases than the RGB one.

First question would be: is that correct, or did I just fall for single people claiming something?

The second, more important question: Is there a way to properly include NIR in the LAB color space? Or would it be better to keep it as separate information?

And lastly, another question that came up: Can I even transform RGB to LAB properly? What information would I need (wavelengths of red, green and blue; some information on the whitepoint, ...)? Is there a different color space better suited?

If relevant: I am focusing on plant images and tasks appearing there, e.g. plant/background segmentation, fruit detection, leave counting,...

$\endgroup$
  • $\begingroup$ Hi @Dirk, and welcome to DSP.SE. I cannot say anything about which color space is better suited for machine learning. Typically, the choice of parameters (such as the color space) should be made for each application individually. Hence, did you try which color space works better for your application? Lastly, maybe you can split your question into multiple related questions, addressing, e.g., the inclusion of the NIR in the LAB color space and the transformation from RGB to LAB, so each question gets read by the right people that have expertise with the specific topics. $\endgroup$ – applesoup Jul 30 at 10:09
  • $\begingroup$ There are standard methods for converting RGB to LAB. They won't take NIR into consideration. LAB itself can't really consider NIR. LAB consist of a "lightness" (L) value, a "green to red" (A) value and a "blue to yellow" (B) value. No room in there for NIR. $\endgroup$ – JRE Jul 30 at 11:09
1
$\begingroup$

Let's start with the source of your RGB + near IR images. To break the process down:

  1. Some camera must have taken those images.
  2. The camera sensor is sensitive to light within two given wavelengths. The technical term for this is spectral response.
  3. Alternating elements (Bayer pattern) in a sensor are etched with filters. Each filter type passes certain wavelengths while supressess the others.
  4. The alternating sensor elements accumulate the filtered light from different wavelengths. This creates our very first raw signal. This raw space is then mapped onto a color space, most commonly the sRGB space.
  5. Filter properties are precisely measured by the manufacturers. They use those properties (and assume a D65 white point, unless manually chosen to something else) to convert the captured raw space into sRGB which gets passed down to you (usually with a little compression and anti-aliasing filtering).

Answering your final question first, with the sRGB assumption in place, you are now armed with one very important source of information - your white point is D65 unless explicitly stated elsewhere. You may now safely proceed to convert your sRGB image into LAB. MATLAB has a handy function for the same.

To answer your questions:

First question would be: is that correct, or did I just fall for single people claiming something?

A: People say a lot of things - largely from their personal experience. You must create your own. I personally exect LAB to help. You can read more about that on my Quora answer here.

Is there a way to properly include NIR in the LAB color space? Or would it be better to keep it as separate information?

All traditional color spaces are designed to fall within the locus of human perception. Since near IR is out of bounds for our perception, it's usually modelled as a standalone entity (seperate channel). However, many imaging devices have a near IR spectral response below 750 nm. If that's the case, I would expect statistical correlation between your G or R channels with near IR. A simple PCA would answer that question.

$\endgroup$
  • $\begingroup$ Thanks for your detailed answer. The camera/sensor in question uses flash units in certain wavelengths to collect color information, which are recorded with a "multispectral camera" (and I have no idea what that means, exactly^^). I'm not sure about the white point, but assuming I can get info on the white point and the wavelengths in question, is there a formula to plug this info in to transform it to Lab (or any of the standardized color spaces for now)? Btw, near IR is at 720-750nm in my case. $\endgroup$ – Dirk Jul 31 at 6:43
  • $\begingroup$ Hi @Dirk, sorry about the late reply. I like that you want to do this the right way, but you may be over-complicating this for yourself. Assuming you can get the white point of your scene, a) sRGB to XYZ colorspace, such as here. b) XYZ to LAB space with the desired white point, as described here $\endgroup$ – Rakshit Kothari Aug 6 at 13:41
0
$\begingroup$

My answer focuses on the machine learning part instead of the original question concerning LAB color space.

From a Machine Learning perspective (and more specifically Deep Learning with semantic segmentation networks for your application) taking as input all of the four bands seems optimal unless you can assert that one (or more) channel(s) is (are) not relevant for the segmentation task.

The network is expected to build its own representation of the image and experiments show that firsts layers learn transformations such as color space changers/selectors, blob and edge detectors.

For plant detection green and NIR components are the most discriminant channels (see EGI and NDVI from literature) but others could participate (for ground segmentation for instance).

I suggest training a machine learning model with different inputs (RGB only, B&W, RGB+NIR, EGI, LAB) and then to compare performances between those different approches. You’ll be able to better evaluate the influence of each spectral band in the result.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.