What is the frequency response of binning 2x2 pixels of an image into 1 pixel in software?

Question

What is the frequency response of binning 2x2 pixels into 1 pixel in software? Can the binning introduce aliasing?

Since the Fourier of the 2D boxcar function is a 2D sinc I would intuitively think that the sharp edges of the 2x2 filter would cause ringing in the frequency domain?

Would it not be better to window this filter? E.g. use a 4x4 filter with low weights along the edge and higher weights inside the 2x2 core?

Consider a greyscale image: $I(x_i,y_j)$ where $0 < i < 2*m$ and $0 < j < 2*n$.

Bin together 2x2 pixels:
$$\begin{align} I_b(x_I,y_J) =\ & 0.25\ [I(2x_I,2y_J)+I(2x_I+1,2y_J)+ \\ &I(2x_I,2y_J+1)+I(2x_I+1,2y_J+1)] \end{align} \tag 1$$

where $0 < I < m$ and $0 < J < n$.

The two dimensional frequency of the image before binning: $F(u,v)$.

The two dimensional frequency of the image after binning: $F_b(u,v)$.

What is $$H(u,v)=F(u,v)/F_b(u,v) ?\tag 2 $$

Answer 1

To make sense of this, consider that you are doing two operations: 2x2 averaging, as you express in (1), above, and then decimating by 2 in each direction.

The 2x2 averaging is just filtering. The 2x2 subsampling does, indeed, cause aliasing.

Your (2) is a transfer function, that bears an implicit assumption of a linear and shift invariant process. The 2x2 averaging (without subsampling) is shift invariant, and (2) can apply. Once you've done that averaging, however, the 2x2 subsampling can no longer be expressed as a simple transfer function.

You could recast (1) so that it operates on $x_i$ and $x_j$ (i.e., no subsampling) and find the frequency response; then you could apply the effects of aliasing.

Answer 2

Note that cameras have lenses and lenses have a point spread function (PSF) that behaves like a spatial continuous domain lowpass-filter. Depending on aperture, quality of optics and scene/camera movement during the exposure.

Note also that many sensors have an Optical Lowpass Filter (OLPF) that acts like a continuous spatial domain comb filter tuned more or less to have some attenuation relative to pixel pitch.

Note that sensel active sensing area may not be a perfect square of dimensions inverse to the sensel pitch.

Note that color cameras typically have a color filter array (CFA) that repeats some «red»/«green»/«blue» filter pattern, usually the Bayer pattern on top of the monochrome image sensor:

R G
G B

For color cameras doing binning, the most common CFA seems to be «quad bayer» where the filtering is e.g.:

R R G G
R R G G
G G B B
G G B B

Notice that this quad Bayer CFA is just a 2x2 «upsampled» regular Bayer CFA. Thus, for a simplistic analysis, a 2x2 binned image from a 48 MP sensor may be expected to have similar properties to a 12 MP sensor of similar dimensions, assuming that olpf is scaled accordingly (which is anyones guess) and that the binning is a simple addition.

The analysis could be as complicated as you care to make it (and the characterisation of components)Or you could assume that it is a 2x2 boxcar prefilter prior to downsampling, with the associated poor frequency-domain suppression of the bands that cannot be reconstructed according to Nyquist.

Answer 3

Attempt at Fourier analysis:

Underlying continous image: $I_{cont}(x,y)$.
Introduce comb function: $$C(x,y)=\sum_{k=-\infty}^{k=\infty} \sum_{l=-\infty}^{l=\infty}\delta(x-k\Delta x) \delta(y-l\Delta y) $$

Original image:
$I = I_{cont}*C(x,y)$

$\mathcal{F}(I)=\mathcal{F}(I_{cont})\mathcal{F}(C)$

Binned image: $I_b = 0.25I_{cont}*C+0.25I_{cont}*C_{10}+0.25I_{cont}*C_{01}+0.25I_{cont}*C_{11}$

where $C_{10}=C(x-\Delta x,y)$, $C_{01}=C(x,y-\Delta y)$ and $C_{11}=C(x-\Delta x,y-\Delta y)$

$\mathcal{F}(I_b)=0.25\mathcal{F}(I_{cont})[\mathcal{F}(C)+e^{-i2\pi u\Delta x}\mathcal{F}(C)+e^{-i2\pi v\Delta y}\mathcal{F}(C)+e^{-i2\pi(u\Delta x + v\Delta y)}\mathcal{F}(C)]$

$\mathcal{F}(I_b)=0.25\mathcal{F}(I_{cont})\mathcal{F}(C)[1+e^{-i2\pi u\Delta x}+e^{-i2\pi v\Delta y}+e^{-i2\pi(u\Delta x + v\Delta y)}]$

$H(u,v)=0.25*[1+e^{-i2\pi u\Delta x}+e^{-i2\pi v\Delta y}+e^{-i2\pi(u\Delta x + v\Delta y)}]$