Filter design with constrained impulse response

Question

What I want

I wish to design an LPF, preferably an FIR, but also open to IIR. My constraints:

• I care about phase non-linearity. Thus if I use an IIR, I need a filter similar to a Bessel filter.
• My signal isn't too long, so I wish to have a short transient (short impulse response).
• A seemingly contradictory requirement with respect to the previous one is that I want to use a narrow transition band as possible.

---

What I know

• Generally speaking, for FIR, the transition width is inversely dependent on the filter order (and therefore, on the impulse response length).
• The proportionality constants depend on the design method. For instance, for Kaiser and Parks-McClellan, these formulas are:
  • Kaiser: $M=\frac{-20\log_{10}\delta-8}{2.285\Delta\omega}$
  • PM: $M=\frac{-10\log_{10}(\delta_1\delta_2)-13}{2.324\Delta\omega}$
  • Thus, for the same $\delta$, PM will provide a shorter filter.

---

My questions:

1. Are there design methodologies that prioritize transition width and minimize order, perhaps a the cost of other specifications? Are there design methodologies that are better than others in these specific aspects?
2. For IIR, the impulse response length isn't well defined, and its decay rate isn't as simply related to the filter order as FIR. Are there design methodologies for IIR which allow controlling the rate of decay of the impulse response in IIR?
3. Are there other methods of dealing with the transient effect at the beginning of the signal?
4. If I care for the linear phase, could I be better off using an IIR or a minimum phase FIR and then correcting it with an all-pass? Or would the cascaded filter be worse in terms of overall delay?

Edit

Following the suggestions I designed an initial filter, I used both the suggested least squares approach (firls) and the PM approach (firpm). Some details:

• I'm interested in a passband of width $0.317460317460317\times \pi$ rad/sample. Obviously, the current design is a linear phase.
• Regarding phase non-linearity within the passband, if possible, I would like to remain beneath a ripple of 0.03samples. I'm not sure if I should make the specification regarding the phase or group delay.
• Filter order - I currently design an 18 taps filter, and I would like to remain between 18-36 taps.

More questions

1. SHould I specify requirements regarding phase response, group delay, or phase delay?
2. The current answer suggests going for the least squares solution, but my initial filter shows that for the same transient, I get a narrower transition band with the Parks-McClellan (Chebyshev) approach at the expense of stopband attenuation. Is there a specific reason to go for the LS solution?
3. Is there an analytic approximation for the LS filter (similar to PM and Kaiser) relating filter order and transition width?
4. Eventually, after I have some candidates, I'll have to quantify the tradeoffs between the excess noise (which results from less stopband attenuation/transition width) versus the distortion introduced by the passband ripple and the transient (which distorts initial samples). Any suggestion on how to perform this analysis?

Edit No. 2

Let me first clarify the use of the filter. The filter will be a part of a signal acquisition channel. As such, I do not wish, nor can I determine exactly what processing will follow. I "simply" wish to provide the best signal, although obviously, this isn't simple (as we need to determine what best means) :)

What specs can I provide:

• Impulse response length - up to 18-36 samples. Anything above 18 samples already distorts initial (interesting) samples but is acceptable if the resulting noise suppression is worth it. According to my understanding, this also dictates the mean group delay (at least within passband)
• Group delay variation - preferably smaller than 0.03 samples. Definitely smaller than 0.1 samples.
• Passband of width $f_{pass}=0.317460317460317\times \pi$ rad/sample
• passband ripple - Originally, we went for 1dB.
• Regarding stopband rejection, we originally aimed for 60dB.
• Transition width, we originally aimed for $0.1f_{pass}$.

I do not think all of these can be accommodated simultaneously, as according to fred harris rule of thumb, these specs should require:

$$ M = \frac{2}{0.03}\frac{60}{22} \approx 181\; taps $$

Therefore, if I'm willing to relax my requirements regarding ripples and transition width but not those regarding impulse response length and group delay variation, what passband ripple and noise suppression can I expect? With firls I get the below filter. Can it be improved?

Answer 1

Regardless of FIR or IIR, the transition width will be inversely proportional to the length of the impulse response. The frequency response and impulse response are Fourier Transform pairs, so this relationship may be clear by reviewing basic Inverse Fourier Transforms of frequency domain transitions.

The OP mentioned a shortest possible impulse response is important and I assume that includes overall delay. If so, a linear phase FIR filter that will satisfy any phase non-linearity concerns will have a delay that is one less than half the number of coefficients. At the other extreme for an FIR is a minimum phase FIR which would have the least delay for any given magnitude response in frequency, but at the cost of phase non-linearity. Cascading a minimum phase FIR with an all-pass to be linear phase IS the resulting linear phase filter (nothing gained).

Consider using the least squares algorithm (firls in Matlab, Octave and Python scipy.signal) to design an optimal (in the least squares sense) linear phase filter based on a desired maximum acceptable transition width (at the minimum possible sampling frequency). Set the number of coefficients to meet target passband ripple and stop-band rejection requirements which will result in the best that can be achieved for a linear phase FIR solution.

If the solution from doing the implementation above is not acceptable, it's very possible this can be improved further with an equalized IIR filter, but for that I recommend starting with quantified specifications on all parameters that are important to understand the trade space (how bad can the phase non-linearity be, how bad can the delay be, how band can the frequency transition be, how bad can the filter rejection in the stopband be, etc). Something has to give to deviate from the linear phase (no phase distortion) solution, and any flexibility on any of those parameters will lead to improvement on whatever is most important. With such specs provided, doing the firls comparative solution will be very quick; I can provide the result here from which others can try to improve on with alternate approaches (which would be very interesting!).

Update to address added questions

1. Should I specify requirements regarding phase response, group delay, or phase delay?

Group delay variation is a good way to specify phase non-linearity over frequency (given it is by definition the negative of the derivative with respect to frequency). Also clarify if you care about both variation (non-linearity) and the overall delay itself; in which case both parameters should be specified.

2. The current answer suggests going for the least squares solution, but my initial filter shows that for the same transient, I get a narrower transition band with the Parks-McClellan (Chebyshev) approach at the expense of stopband attenuation. Is there a specific reason to go for the LS solution?

If the stopband attenuation was better, then you could just tighten the target transition band on least squares to exceed the Parks-McClellan solution until the average stop band performance matches. Both solutions are viable, they are just "optimum" under different constraints. In most filtering applications I personally work on, having less rms error overall is superior to having less peak error. Least squares will give you the "optimum" solution for minimum rms error, while P-M gives you the optimum solution for minimum peak error (where optimum is with regards to the error from the target performance of passband ripple, transition and stop band performance for a linear phase solution, which has zero group delay variation; it may not necessarily be optimum for other considerations). So in this case if you were to adjust the designs to have the same transition band (for the same number of taps), you will find small areas in vicinity of the transition band where the least-squares solution is worst, but then everywhere else across both passband and stopband, the least-squares will be much better. So unless I am working from an absolute "thou shall not exceed" spec applicable to the entire spectrum, I favor the least-squares algorithm for linear-phase filter design. (Note, as mentioned in other posts here, the windowing approach to FIR design using the Kaiser window often comes quite close to least-squares performance, so that is my next go-to when least-squares doesn't converge or computing dynamic filters on the fly with minimum resources).

3. Is there an analytic approximation for the LS filter (similar to PM and Kaiser) relating filter order and transition width?

Yes, see this post. (And they are simply starting approximations not exact soltutions). Since the Kaiser windowed filter results so closely in least squares, that approximation is also fine to use.

4. Eventually, after I have some candidates, I'll have to quantify the tradeoffs between the excess noise (which results from less stopband attenuation/transition width) versus the distortion introduced by the passband ripple and the transient (which distorts initial samples). Any suggestion how to preform this analysis?

That could be application specific as you may have more concern for noise in different portions of the band (or not). Ultimately your filter is required to meet some externally observable performance specification for your product, so I would work back from that. You also may have other considerations such as power, cost, size impacts (or not).