Understanding the Bilateral Filter - Neighbors and Sigma

Question

I have been reading about the Bilateral Filter in - Fast Bilateral Filtering for the Display of High Dynamic- Range Images and have felt crippled and confused while understanding its working and the parameters.

In the equation for Bilateral Filter given as

• How to determine the number of neighboring pixels?
• Are all adjacent pixels the neighboring pixels? (top, right, bottom, left, diagonal)
• What does it really mean when people say the following (underlined):

Please help me understand this. I feel lost as I try to understand and gain an intuition of the Bilateral Filter.

I plotted a graph of the Gaussian function (y-axis represents the value of Gaussian function;) with three-sigma values 0.2, 0.5 and 0.9 to help me understand but got nothing.

(blue = 0.2, green = 0.9)

Answer 1

I would start with the many resources on this site:

• Is the Bilateral Filter a Solution of Some Variational Method?
• How to Validate Bilateral Filter Implementation?
• Comparison Between Guided Filter (Edge Preserving Filter) and Gaussian Filter.
• What Is the Bilateral Filter Category: LPF, HPF, BPF or BSF?
• Understanding the Parameters of the Bilateral Filter.

Regarding your questions, Let's address them one by one.

Q: How to Determine the Number of Neighboring Pixels?

The classic neighborhood used by Bilateral Filter is along the axis. So it is basically determined by Radius parameter. In order to make sense, the radius parameter is aligned with the $ {\sigma}_{s} $ parameter (Spatial Standard Deviation). Some examples of the connection between them can be found in:

• How to Get Rid of Ripples from a Gradient Image of a Smoothed Image?
• Parameters of Gaussian Kernel in the Context of Image Convolution.
• How to Calculate Gaussian Kernel for a Small Support Size?
• Gaussian Blur In MATLAB and Connection to Image Resolution.

Q: Are All Adjacent Pixels the Neighboring Pixels?

Well, Theoretically you can chose any way to set the neighborhood set (Circle, The whole image, etc...). But as I wrote above, the most common option it a rectangle (Square in most cases).

Q: What Are the Parameters $ {\sigma}_{s} $ and $ {\sigma}_{r} $ For?

The parameters determines the range of values in the range and spatial domain.
Let's assume our image values are in the range [0, 1] and the spatial indices are integers. Assume $ {\sigma}_{r} = 0.1 $, then pixels with values which are farther than $ \sim 3 {\sigma}_{r} $ will have effective zero weight since $ {e}^{ -\frac{ {0.3}^{2} }{ 2 \cdot {0.1}^{2} } } = 0.0111 $ before normalization.

Same logic for the Spatial parameter $ {\sigma}_{s} $. If we set it to $ {\sigma}_{s} = 4 $ pixels which are more than 12 indices apart will have negligible weight. This is why we limit the radius of the neighborhood and match it to the spatial parameter as adding pixels which have zero weight is waste of computing resources.

Answer 2

Taking a step back, a standard image $I$ is composed of pixels. They are defined by a location (spatial coordinates, here denoted by $p$), and a "value", denoted by $I_p$. Smoothing or enhancing an image, in a large sense, consists in replacing each $I_p$ by $\hat{I}_p$, a value that would be: more probable, more consistent, more visually pleasant (choose your favorite).

This job can be performed using pixels, models and functions to combine them. And assumptions on what you deem probable, consistent, pleasant... A classical assumption is that a noisy or erroneous pixel value could be replaced by combinations of other pixel values. Which pixels is the central question? Two answers are standard:

• pixels close in location to $p$, giving most local filters, often with finite support,
• pixels close in intensity, giving non-local filters (like Non-Local NL-means).

with a weight to modify each pixel's influence in the result. To simplify the discussion, let us assume that the combination of pixel values is a weighed mean (but it could be any generalized mean, median, etc.). Generally, the closer the value, the higher the weight. The bilateral filter combines these two options. A smoothing weight is generally chosen so that the sum of weights is one (Question 3). Thus, a constant image is invariant to smoothing. This settle the question on $W_p$. Then, given a pixel location $p$, one can choose how the selected pixels $q\in S$ are picked, in the set $S$: which regions, etc. You can choose to have only pixels in the corners, on the borders etc. if you want. You can choose only left-right-top-bottom pixels, or diagonal, as you want (Question 2).

This depends on the prior you have on their validity. On top of that, you can choose how those pixels act, on their position $q$ relative to $p$. This is driven by $\|p-q\|$, where you can choose a suitable norm. With $G_{\sigma_S}(\|p-q\|)$, the term $\sigma_S$ sets a decaying Gaussian function around $p$, providing an implicit neighbor size, depending on the norm choice for $\|\cdot\|$ and $\sigma_S$. So far, this is a traditional linear Gaussian image filtering. The bilateral filter adds a non linearity, as $|I_p-I_q|$ depends on image values. It tells you: how do intensities of pixels, already in the neighborhood of $p$, further contribute to the average (Question 3). This ingredient is useful to preserve edges. For instance, take an edge separating a white and a black region. Take a black pixel near to the edge. A black pixel in its neighborhood will have more weight than a white one, because of $|I_p-I_q|$, which is not the case in standard linear filtering.

For Question 1, the choice really depends on the scale of objects ($\sigma_S$) and the range of intensities ($\sigma_r$) you want to preserve, and this is domain dependent. For natural images, there must be parameters given in papers linked to by Royi.