How do I deal with pixel intensity overflow when performing discrete convolution on an image and the result is greater than 255?

Question

This is my first post on this forum, and I have been working to come up with a solution for a final project in my hardware design class using system verilog on an Altera Cyclone FPGA. I've been having trouble finding a solution for overflow when convolving a 3x3 pixel intensity matrix (RGB, values from 0-255) with a specific kernel (blurring, canny edge detection, embossing, etc.)

The question I have is this: What should I do to deal with a result that is greater than 255? I've seen some references suggesting to "wrap the values around" from 255 back to 0 and up, and also have seen some references suggesting to multiply all pixels by a "shrinking factor" (e.g. .8) to avoid ever having to deal with pixel intensity overflow.

Sadly, I've found few references explaining how to deal with this issue in hardware realization. Am I missing something here? Any assistance would be greatly appreciated.

EDIT:

Specifically, what if you convolved a matrix of

[255, 255, 255; 255 255 255; 255, 255, 255] * [1, 1, 1; 1, 1, 1; 1, 1, 1]

? Wouldn't your result contain values outside of your range of 0-255?

Answer 1

There is no perfect solution to the problem you face. You must make a choice and every choice has pros and cons. For example you can apply saturation thresholding. Or you can use normalization. Saturation threshold clipping is very simple to realize and provide excellent performance provided that intermediate processing stages will not magnify the intensities too much above 255 (or below 0). Otherwise, it will severely distort the image. Normalization guarantees a loss free output but may severely degrade the algorithm quality. Besides it requires you to know in advance the normalization scale, which may not be possible always.

Answer 2

Unfortunately I'm not too familiar with the hardware so I can't provide a much more informed answer than this. I think normalization would be the most straightforward solution, as you can divide all values by 255 (the known largest value), perform your operations, find the new maximum value, then scale back up afterward so that your maximum value is 255 again. This, I think, should work for any operation you want to do, as long as you can keep the flating-point precision required to not lose any information about your original signal.

Of course, if beforehand you do know what kind of operations you need to apply, then you can do something else. For example, you can just multiply by a constant like 0.8 (as you suggested), if you know that your values aren't going to be terribly large. Or, if you're performing convolutions, you can scale your kernel down before convolving.