Design of the low pass filter when decimating

Question

I am filtering an audio signal into various sub-bands in C and would like to decimate the signal by 2 in the lower frequency bands to reduce processing overhead. I get that this involves low-pass filtering the signal and then throwing away every other sample, but how do I go about designing the low-pass filter optimally?

At the moment I have a signal sampled at 44.1 kHz. Say that for signals in bands limited below 8 kHz, I low-pass filter with a 2nd order butterworth filter with a cutoff of 9 kHz and then decimate. This works fine as I filter well below the new Nyquist limit of 44100/4, but it feels very unscientific.

What is the best filter type for this purpose. Looking at the matlab documentation, the decimate function uses an 8th order Cheby Type 1 filter by default, but I have no idea why. Is there some rule-of-thumb that I'm missing?

Answer 1

You can determine the "best" filter by analyzing what type of errors is created and how much of each error your specific application can live with.

If your original content is already perfectly band limited, than you don't need a low pass filter at all and can directly throw away the extra samples.

The decimation low pass filter typically determines how the band limiting of the content is done. Here are the factors to consider

1. Aliasing: since the low pass filter is never perfect, the decimation will create some amount of aliasing. The amount depends on the steepness of the filter and the energy content of the signal at the frequencies that need removing.
2. Pass band: The low pass filter isn't perfectly flat in the pass band so there is some amount of ripple you need to live with. The pass band must be below the new Nyquist frequency. The higher the filter order, the closer you can get and the wider the pass band will be.
3. Phase response: Any IIR filter will change the phase response of your signal. The steeper the filter, the higher the maximum group delay. If need to maintain the phase, you need to use a linear phase FIR filter (ideally in a polyphase configuration).

The choice of filter is a trade off between pass band width, pass band amplitude distortion, pass band phase distortion, aliasing noise and computational complexity.If you have a spec for all of these, you can design the "best" filter. An example would be "pass band +-0.5dB up to 90% Nyquist, aliasing at -80dB or below, linear phase".

Answer 2

Every type of filter has its characteristics. For instance, Butterworth filter needs a higher order than Chevyshev, which implies that it is likely that it would computationally cost more. Other characteristics, such as the ripple in the stop filter (which implies different attenuation in those bands), the phase distortions in both stop and pass bands, filter complexity, etc. may influence your choice.

Personally, I would make it clear what the limits for all of these parameters (prioritized or weighted parameters) would be before I try to choose any kind of filter. Then, I would design several of them (of many kinds) complying with those limits and compare the filters using those parameters. To choose one of two or more filters like, I assign a weight or priority to each characteristic of the above, weigh each filter by its nature, and see which one wins. That is a scientific method.