0
$\begingroup$

I have a camera which is providing me raw, binary data with 10 bit data. It follows the Bayer pattern RGGB, and each pixel is saved using 16 bit chunks. I am able to demosaic the image almost successfully.

The problem

The RGB image looks is almost completely black. If I multiply by 64 the raw data, the RGB image looks better but rather green. The overall intensity is very low. The camera provider has a live view of the camera and it looks much better in both aspects.

My guess

Since 10bit are saved into 16bit chunks, the tools I am using assume 16bit depth (at least Matlab and OpenCV do for sure). In order to compensate for this, I multiply by 64 (6bit shift to the left) the raw data. But still is not enough. If I multiply by 128, some pixels saturate and then the image looks weird.

What I have tried

Using Matlab image processing toolbox, Python colour-demosaicing package, and OpenCV functions, I get the same result, but they do not match the live visualization.

Update I analyzed the histogram on the vendor tool, and those from OpenCV. I am not sure, but interestingly there are no values above 255. However, on the raw data I record from their software, there are many values above 255, but nothing compared with values below 255. Thus, if I clip the image with max value of 255, the final result looks very similar!.

Questions

  • Is there anything else I can verify? I assume is a problem on my end, because the three tools give same results.
  • Is there a better way of handling the 10bit pixels to convert to 16bit?
  • Is there any sense on what I discovered on the update? It seems that the two extra bits are not used for anything but noise!

Any help is appreciated. I am very sorry for not providing data or images (company privacy).

Thanks!

$\endgroup$
  • $\begingroup$ Big endian Little Endian issue? $\endgroup$ – Stanley Pawlukiewicz Apr 14 '18 at 2:41
  • $\begingroup$ Please add your input image and your code! $\endgroup$ – Balaji R Apr 14 '18 at 4:34
  • $\begingroup$ @StanleyPawlukiewicz thanks for the suggestion. Everything is little endian and I am sure I am reading it properly because the first 4 bytes of the data are 2 uint16 specifying the image resolution, and those numbers are correct. $\endgroup$ – Javi V Apr 15 '18 at 2:07
  • $\begingroup$ @BalajiR I am afraid I cannot due to privacy policies :( $\endgroup$ – Javi V Apr 15 '18 at 2:07
1
$\begingroup$

The raw data you get from the camera is nearly linear in the colorspace spanned by the camera CFA filters. The 8-bit/sample monitor you have assumes the image is gamma-compressed, most likely within the sRGB colorspace. These two colorspaces are different, therefore you need to map between the two in some way. Understanding this, lets address each of your issues:

The RGB image looks is almost completely black.

The maximum 10-bit value is 1023, the maximum 16-bit value is 65535. To Convert 10-bit linear to 16-bit linear you multiply by 65535./1023 and round to nearest integer. Alternatively it might be easier to work with floating point directly, so just divide by 1023 to map your data to the [0,1] range.

the RGB image looks ... rather green

It is common for the camera sensors to be more sensitive to green, so the green response is higher than red and blue, giving an overall green tint. To get physically correct measurements of the incoming light within a known colorspace (say the ITU-R BT.709 primaries used by your monitor) you need to figure out the color transformation matrix from the camera colorspace to your preferred colorspace. There are different ways to accomplish this: get a color-calibration chart, contact the sensor manufacturer, or measure the exact spectral response using other means (like a monochromator).

That being said, if you don't seek for physically correct measurements but rather just for a good-enough picture, you might skip the above calibration step and instead do only a white-balancing step: find an ought-to-be gray object in the frame and divide each pixel by the RGB of that object, thus cancelling the green tint. This on its own, without a color transform, will not reproduce colors exactly though.

The overall intensity is very low

This is due to the linear versus gamma discrepancy. You do all your math in linear color space with high bitness, and as a last step, convert it to 8-bit sRGB for display. See sRGB for the exact formula.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.