# What is the frequency response of binning 2x2 pixels of an image into 1 pixel in software?

What is the frequency response of binning 2x2 pixels into 1 pixel in software? Can the binning introduce aliasing?

Since the Fourier of the 2D boxcar function is a 2D sinc I would intuitively think that the sharp edges of the 2x2 filter would cause ringing in the frequency domain?

Would it not be better to window this filter? E.g. use a 4x4 filter with low weights along the edge and higher weights inside the 2x2 core?

Consider a greyscale image: $$I(x_i,y_j)$$ where $$0 < i < 2*m$$ and $$0 < j < 2*n$$.

Bin together 2x2 pixels:
\begin{align} I_b(x_I,y_J) =\ & 0.25\ [I(2x_I,2y_J)+I(2x_I+1,2y_J)+ \\ &I(2x_I,2y_J+1)+I(2x_I+1,2y_J+1)] \end{align} \tag 1

where $$0 < I < m$$ and $$0 < J < n$$.

The two dimensional frequency of the image before binning: $$F(u,v)$$.

The two dimensional frequency of the image after binning: $$F_b(u,v)$$.

What is $$H(u,v)=F(u,v)/F_b(u,v) ?\tag 2$$

To make sense of this, consider that you are doing two operations: 2x2 averaging, as you express in (1), above, and then decimating by 2 in each direction.

The 2x2 averaging is just filtering. The 2x2 subsampling does, indeed, cause aliasing.

Your (2) is a transfer function, that bears an implicit assumption of a linear and shift invariant process. The 2x2 averaging (without subsampling) is shift invariant, and (2) can apply. Once you've done that averaging, however, the 2x2 subsampling can no longer be expressed as a simple transfer function.

You could recast (1) so that it operates on $$x_i$$ and $$x_j$$ (i.e., no subsampling) and find the frequency response; then you could apply the effects of aliasing.

• Thank you for your help! Does this mean that there does not exist a general frequency response function? I will have to calculate one per image? Also I do not understand how I can recast (1) so that it operates on i and j indexes. This sounds like a convolution with a filter and I do not think that this is what is happening. The filter is only applied at a subset of (i,j).
– Andy
Sep 14 at 6:53

Note that cameras have lenses and lenses have a point spread function (PSF) that behaves like a spatial continuous domain lowpass-filter. Depending on aperture, quality of optics and scene/camera movement during the exposure.

Note also that many sensors have an Optical Lowpass Filter (OLPF) that acts like a continuous spatial domain comb filter tuned more or less to have some attenuation relative to pixel pitch.

Note that sensel active sensing area may not be a perfect square of dimensions inverse to the sensel pitch.

Note that color cameras typically have a color filter array (CFA) that repeats some «red»/«green»/«blue» filter pattern, usually the Bayer pattern on top of the monochrome image sensor:

R G
G B


For color cameras doing binning, the most common CFA seems to be «quad bayer» where the filtering is e.g.:

R R G G
R R G G
G G B B
G G B B


Notice that this quad Bayer CFA is just a 2x2 «upsampled» regular Bayer CFA. Thus, for a simplistic analysis, a 2x2 binned image from a 48 MP sensor may be expected to have similar properties to a 12 MP sensor of similar dimensions, assuming that olpf is scaled accordingly (which is anyones guess) and that the binning is a simple addition.

The analysis could be as complicated as you care to make it (and the characterisation of components)Or you could assume that it is a 2x2 boxcar prefilter prior to downsampling, with the associated poor frequency-domain suppression of the bands that cannot be reconstructed according to Nyquist.

• There may be other binning schemes out there working on regular Bayer CFAs by combining non-immediate neighbours of the same color. That sounds «fiddly» both from an electronics as well as dsp-analysis PoV Sep 14 at 6:54
• Are you saying that most likely the image is already low pass filtered so much that 2x2 subsampling does not cause aliasing?
– Andy
Sep 22 at 13:11

Attempt at Fourier analysis:

Underlying continous image: $$I_{cont}(x,y)$$.
Introduce comb function: $$C(x,y)=\sum_{k=-\infty}^{k=\infty} \sum_{l=-\infty}^{l=\infty}\delta(x-k\Delta x) \delta(y-l\Delta y)$$

Original image:
$$I = I_{cont}*C(x,y)$$

$$\mathcal{F}(I)=\mathcal{F}(I_{cont})\mathcal{F}(C)$$

Binned image: $$I_b = 0.25I_{cont}*C+0.25I_{cont}*C_{10}+0.25I_{cont}*C_{01}+0.25I_{cont}*C_{11}$$

where $$C_{10}=C(x-\Delta x,y)$$, $$C_{01}=C(x,y-\Delta y)$$ and $$C_{11}=C(x-\Delta x,y-\Delta y)$$

$$\mathcal{F}(I_b)=0.25\mathcal{F}(I_{cont})[\mathcal{F}(C)+e^{-i2\pi u\Delta x}\mathcal{F}(C)+e^{-i2\pi v\Delta y}\mathcal{F}(C)+e^{-i2\pi(u\Delta x + v\Delta y)}\mathcal{F}(C)]$$

$$\mathcal{F}(I_b)=0.25\mathcal{F}(I_{cont})\mathcal{F}(C)[1+e^{-i2\pi u\Delta x}+e^{-i2\pi v\Delta y}+e^{-i2\pi(u\Delta x + v\Delta y)}]$$

$$H(u,v)=0.25*[1+e^{-i2\pi u\Delta x}+e^{-i2\pi v\Delta y}+e^{-i2\pi(u\Delta x + v\Delta y)}]$$