# Derivation of a non-ideal low-pass rectangular windowed FIR filter

I have been trying to understand certain aspects of FIR filter design which have frankly annoyed me for some time such as exactly why the critical frequency $$\omega_c$$ in a low-pass FIR filter is mapped to the $$-6\;\text{dB}$$ point for a non-ideal low-pass FIR filter and also why the critical frequency is mapped to point halfway between the transition band $$\omega_t$$. Below I will show my logic up until now.

If we take the ideal low-pass filter in the frequency domain, $$H_{ideal}(\omega)$$, with the critical/crossing frequency $$\omega_c$$, and take the inverse discrete-time Fourier transform, we get some response $$h_{ideal}(n)$$.
If we window this function with a rectangular window for simplicity but it could be any other window $$w(n)$$, we get the following time-domain result: $$h(n) = h_{ideal}(n) \times w(n)$$ As we know, multiplication in the time-domain is convolution in the frequency domain and vice-versa, thus $$H(\omega) = H_{ideal}(\omega) * W(\omega)$$ This analytic solution gives all of the properties of the filter, such as for a window of length N, giving N coefficients, how can we derive the attenuation, the location of the shifted critical frequency and the length of the transition band.

I have tried to solve this convolution before and gave up as I essentially found myself in an infinite integration by parts loop. If the analytical solution is perhaps too complicated, could anyone point me in the direction of how the properties of a filter with a particular window are actually derived ?

• "The result of this convolution [...] transition band.". This is confusing. I think you should rephrase: "This analytic solution gives all of the properties of the filter, such as for a window of length $N$, giving $N$ coefficients, how can we derive the attenuation, the location of the shifted critical frequency and the length of the transition band"
– Jdip
Nov 9, 2022 at 13:39
• @Jdip Thanks for the recommendation and the changes should be reflected in the question. Nov 9, 2022 at 14:17

The properties of a filter designed using the window method are dependent on two parameters:

• The length of the window
• The design window function

Long story short:

• The transition bandwidth is approximately equal to the window's main lobe width, which is itself dependent on the window function AND window length. For example, a rectangular window of length $$M$$ has main side lobe width $$\dfrac{4\pi}{M}$$ (including the negative half, see @RickarySanchez's comment), whereas a hamming window of length $$M$$ has main side lobe width $$\dfrac{8\pi}{M}$$. For the same length, the rectangular window hence has a sharper transition width.
• The attenuation depends on the window's side lobe height, which is itself dependent on the window function. For example, a rectangular window of length $$M$$ has first side-lobe height at $$-13\,\text{dB}$$ while a hamming window of length $$M$$ has first side-lobe height at $$-41\,\text{dB}$$. For the same length, the hamming window will therefore have better attenuation in the stop-band.
Less important but worth noting: for a given window, the attenuation also depends on the transition bandwidth (the sharper the transition, the better the attenuation).
• For the cut-off $$\omega_c$$ to be at $$-6 \,\text{dB}$$, the transition bandwidth needs to be sharp enough. An illustration can be found in a previous answer of mine.

• I'm pretty sure the width of the main lobe of a rectangular window is $\frac{2 \pi}{M}$ radians/sample or $\frac{4 \pi}{M}$ including the negative half Nov 9, 2022 at 14:47