What is the difference of windowing functions for FIR filtering?

Question

What is the difference of window types such as Hann, Rectangular, Hamming in the application. Is there such a thing that the one is better than the other?

Answer 1

So, from the discussion in the comments it's clear you know most you need to know.

The window method for FIR filter design is based on this idea:

1. We know the "ideal" frequency response $H(f)$ we want. Often, that's something like a rectangle in frequency design.
2. Well, the easiest thing to achieve that shape would simply be transforming $H$ to time domain, yielding the impulse response $h(t) = \mathcal F^{-1}\{H\}(t)$.
3. small problem here: If $H(f)$ is very steep, then the properties of the Fourier transform implies that $h(t)$ will be very long – in fact, if $H(f)$ is a rectangle (we call that the "brickwall" or "boxcar" filter, by the way), then $h(t)$ will be a sinc; and that is infinitely long. That's not useful for a FIR....
4. So we decide we need to cut off $h(t)$ after some length. Now, if we simply decide to cut off at some point, we have multiplied with a rectangular window $w_\text{rectangle}$, and we know that this is equivalent with convolving with a sinc in frequency domain. And convolving our nice ideal frequency response $H(f)$ with a sinc will totally "smear" and make it round and wobbly – not really what we want, usually.
5. Therefore, instead of using the rectangular window, we use a "rounder" window.
6. Thus, we realize we need to find a trade-off between being "sharp" in frequency and not "too smoothly running out" in time domain (because that means our filter remains long).

Now, the filters you mention have different properties, and were all designed to fulfill some roles.

For example, the Hamming window has a very nicely suppressed first sidelobe in frequency domain – which means that after convolving our ideal $H$ with it, the transition from passband to stop band will show the same nice behaviour, and signals close to the edge of the passband are already well suppressed. That might be useful when you want to extract a single channel from many channels that lie close together in spectrum.

Hann, on the other hand, doesn't suppress the first sidelobe as much - but in exchange, the power of the sidelobes drop faster the further you get from the mainlobe. This helps you if you want to design e.g. an anti-aliasing filter, where all the frequency that you don't suppress ends up aliased in the passband.

Now, for your oral exam: Maybe head over to wikipedia's list of window functions, and look at the frequency domain plots of them. When your examiner asks you what the up- and downsides for a given window for a given application are, you'd start by explaining that you need to convolve the ideal frequency response with the spectrum of the window, and then work your way towards an answer.

If your examiner is like any of the examiners that I was the transcript writer for (or any examiner that orally examined me when I was still a student), then that ability to work towards a solution based on an understanding of windowing and an analysis of what the application calls for is far more interesting than the ability to have an answer to the question "What window will you use for designing an anti-aliasing filter?" instantly ready.