bias-variance trade-off in image denoising

Question

I am reading the paper A Bias-Variance Approach for the Nonlocal Means.
One sentence from the paper is as follow:

To discuss the tuning of parameters of the NLM, we interpret this choice as a bias-variance dilemma.

I encountered bias-variance for the first time in papers on image denoising and I'm not sure about the meaning of bias-variance. Does the bias mean the difference between the original image and denoised image?
What does variance mean? Does it mean some measurement about the denoised image?

What I guess is:
For the original image, the bias is smallest and the variance is largest.
For the denoised image with every pixel having pixel value of mean of the original image, the bias is largest and the variance is smallest.
How can this guide to design de-noising method or tuning parameters?

Answer 1

Bias usually refers to some tendency of the estimator to give a particular answer.

For example, suppose we want to estimate the current time of day, $\hat{T}$.

One such estimator could be: $$ \hat{T} = 10 \mbox{ am}. $$ This estimate perfect at 10am in the morning, but at 10pm in the evening, it's not very good. It's biassed regardless of when you're asking for the estimate.

The paper you link to says that the risk of denoising the pixel $x$ is \begin{align} &\mathbb{E}| N L \tilde{u}(x) - u(x)|^2 \\&= \mathbb{E}| N L \tilde{u}(x) - NLu(x)|^2 \tag{variance}\\ & + \mathbb{E} | N L u(x) - u(x) |^2 \tag{bias}\\ & + 2\mathbb{E}((NL\tilde{u} - NLu(x))(NLu(x) - u(x)))) \tag{goes to zero} \end{align} where $u$ is the original image and $\tilde{u}$ is the noisy image.

The first term on the right-hand side is the variance. This is the variation due to the noise.

The second term on the right-hand side is the bias. This is the deviation from the original image due to the processing $NL$.

Both these terms cause the estimate to be different from what we want.

For the original image, the bias is smallest and the variance is largest.

For the original image, there is no bias and there is no variance.

The variance only comes in when the original image is corrupted by noise.

The bias only comes in when the (original) image is processed.

The original image is not corrupted by noise, nor has any processing been applied yet.

How can this guide to design de-noising method or tuning parameters?

This is really going to depend on what you're using the image for. Some high-bias artifacts might be acceptable.