Is there a use for a changing kernel in image convolution

Question

I understand the principle of image convolution. Maybe. I think it is this:

You use a small matrix with numbers in it, then take a sample of an image with as many pixels as you have numbers in your matrix, then do a dot product, then add up all the terms, and then use the result as the new value for the pixel located in the center of your image sample.

Now, my question is this:

Does it make sense to think about an image kernel for convolution whose values actually depend on the image? For example, consider a 'sharpen' kernel:

0 1 0
1 4 1
0 1 0

I want to change the value in the center to be a number between 1 and 5 depending on the brightness of the pixel which is two places to the left.

The reason I ask this is because I've noticed that when I look at an image, I actually look at 'neighborhoods'. not just individual pixels. While bright spots might catch my attention. I recognize parts of an image before it 'clicks'. So if I am using some parts of an image to inform me about the 'neighborhood' that part lives in, I am wondering if this sort of 'dynamic kernel' could produce a matrix that informs us more meaningfully about the image. Obviously I am naive and know very little about image processing but a lot of papers on image convolution get technical very fast and then I'm lost.

Answer 1

Yes there are image filter kernels whose coefficients may change with location of the processing, and further based on the local image data.

One prime application of such filtering is the edge adaptive, noise reduction where the kernel coefficients are determined based on a statistical analysis of a local group of pixels; according to whether it's an edge dominant or noise dominant block, filter coefficients will change.

Answer 2

A convolution is, by definition, linear and shift-invariant. If the kernel changes according to the location in the image, then the filter is no longer shift-invariant, and therefore not a convolution.

When the kernel changes based on image content, then the filter is no longer linear. It is possible to construct such a filter that is shift-invariant, and these filters are terribly useful. But being nonlinear, these are not convolutions either.

There are many examples of nonlinear shift-invariant filters: the median filter, the trimmed mean filter, the bilateral filter, all the filters from Mathematical Morphology, Frangi’s vesselness filter, etc. etc. etc. Purely linear filters are rather limited, and few image processing tasks can be accomplished with only linear filters.