If I have a noisy signal which I know is approaching the saturation limit of my sensor, then the measurement will sometimes be saturated. It seems that if I try to estimate the true data by averaging the noisy measurement, then the mean will be systematically shifted from the true value (assuming zero mean white noise).
In the example below, when the signal approaches the saturation limit of 100, the moving average is systematically below the true signal. The moving median would solve this particular problem nicely, but I would like to avoid using a median filter for other reasons*.
Is there any "standard" or well known way of dealing with this kind of systematic error which can be expected in advance? (Apart from median filtering.)
I'm interested in the general case, but in case it helps to clarify things, I will include my specific application. I am trying to categorise the brightness of various regions in an image into discrete levels. For black regions, saturation is at 0. Due to noise, most pixels are
>0, none are
<0, so the average is
>0, which sometimes fools my routine. Over the rest of the dynamic range, I can differentiate quite well between N and N+1 levels of brightness. Differentiating between the 1st and 2nd level seems to be more challenging, because of the saturation issue. In the dark (not black) regions, the noise is symmetrically distributed, so I guess I could test for skewness? But I think this would not be very robust for a small number of pixels.
*the other reasons being that I'm actually using a weighted average or Gaussian filter to take advantage of spatial structure within the image. I am not aware of a "weighted median" filter, and I'm assuming at this stage that such a thing doesn't make a whole lot of sense.
new_v = round(LEVELS * old_v/MAX_V)? Where
new_vis the new pixel value,
old_vis self explanatory,
MAX_Vis the maximum depending on colour depth and
LEVELS << MAX_V. You are basically remapping the range to
0..LEVELSwith the quantisation offered by
floorif it suits better your application). This however will not rid you of the clipping. $\endgroup$
LEVELSis the range of
MAX_Vis the range of
old_v. This will reduce the colour depth from $log_2(MAX\_V)$ to $log_2(LEVELS)$. Maybe I am misunderstanding something too here though (?). $\endgroup$