Filter Design - Quantifying the Tradeoffs

Question

Background

I've been given the task of designing a filter. The problem is constrained in terms of filter order, and as such, the specs initially conceived for the filter seem challenging, and we think we might need to release them somewhat. For those interested, here is a question I posted regarding the design requirements.

I aim to quantify the performance of various filters, each making sacrifices regarding the original specs to make the best choice.

My Question

My question now is more general: How would I quantify the performance of a filter? Is there a standard way of doing this? Does it make sense to talk about the SNR of a filter?

My initial thoughts:

• The filter itself doesn't introduce noise but introduces distortion, both in terms of magnitude and possibly phase (for non-linear phase filters).
• I think it doesn't make sense to talk about the "SNR of a filter," as essentially, this doesn't capture the actual noise that can pass through the filter.
• I would assume that looking at the SINAD (signal to noise and distortion ratio) ratio at the filter output to compare between filters makes the most sense.
• However, even though my system is designed to operate at known noise levels, this analysis will depend significantly on the signal used. I wish not to assume a specific PSD aside from its bandwidth.
• Lastly, the suggested analysis ignores the distortion the first samples undergo due to the transient of the filter and considers only the steady state. Is there a way to include this in the analysis?

I would love to get either suggestions/references to standard approaches for quantifying filters or to hear suggestions regarding my current path.

Edit

Following the answer given, I'll add some details. I wish to design a digital LPF.

• The requirements regarding passband ripple, stopband rejection, and transition width imply a filter that is too long for our desire, at least when using standard design approaches such as least-squares, equiripple, or windowing designs. IIR designs weren't considered as of now.
• The filter is supposed to run over quite powerful hardware, so resources are less of a concern, but fixed-point numerical issues are.
• Phase non-linearity is a concern; thus, only linear phase FIRs were considered for now.
• The overall impulse response length of the filter is the main constraint limiting the frequency domain performance as I wish to minimize the transient.

With these in mind, I aim to quantify the performance of various designs in terms of distortion and noise rejection.

Initial Approach

My initial approach is to pass signals through the filter and estimate the noise and distortion as the difference w.r.t to the ideal output, i.e., given an input signal $s[n]$ and output $y[n]$, define the error $e[n]$:

$$ e[n] = y[n] - y_{ideal}[n], $$ Where:

$$ y_{ideal}[n] = \begin{cases} s[n] & s[n]\text{ contained in passband}\\ 0 & s[n]\text{ contained in stopband}\\ \end{cases} $$

I can then try to quantify a given design's quality as the average signal power ratio $\frac{P_s}{P_e}$, averaged over an ensemble of input signals.

Initial Approach Shortcomings

• The suggested analysis will be sensitive to the chosen signal $s[n]$. I can run a frequency sweep over the whole Nyquist range $[0,f_s/2]$.
• However, I wonder what I am missing by not using wideband signals. For sure, I am missing the phase distortion, which is why so far, non-linear phase filters weren't considered.
• The suggested analysis doesn't consider the distortion introduced by the transient (the signal is stationary, and transients are discarded). On a technical level, including the transient distortion into the error isn't hard, but it makes the task of choosing the ensemble of input signals even more challenging.
• The error signal $e[n]$ isn't defined in the transition band. Thus, my analysis will favor designs with a wider transition band on a technicality, i.e. if the spec includes a wider transition band (over which the design incurs no error), it will get a higher score while not necessarily being better behaved.

Any thoughts on how to improve my approach?

Answer 1

Filter design has a complex set of tradeoffs. There is no single "quality metric", the trick is to find the right set of tradeoffs for your specific application. The properties to consider include (but are not limited too)

1. Difference to the "ideal" magnitude response. Even defining this metric is its own can of worms: could be peak, min/max, RMS (linear or dB), weighted, lin/log/other frequency grid etc.
2. Same as 1) but for phase
3. Time domain ringing, preservation of transients
4. Preservation of causality
5. Cost in terms of memory and CPU cycles
6. Latency, real time considerations.
7. Numerical stability and noise
8. Etc.

I would love to get either suggestions/references to standard approaches for quantifying filters or to hear suggestions regarding my current path.

I don't thing there is a standard approach. Offline filtering in Matlab is completely different from a real-time implementation on a resource starved fixed point processor. Causality may matter a lot, it may not mater at all.

The "best" filter is the one the represents the best trade-off between all these properties for your specific problem at hand.

EDIT based on the updated question

The suggested analysis will be sensitive to the chosen signal $s[n]$.

That's indeed the biggest issue here. A better approach would be to calculated the weighted error in the Frequency domain, i.e.

$$E = \sum_{k=0}^{M-1} |H(\omega_k)-T(\omega_k)|^2\cdot W(\omega_k) $$

where $H(\omega)$ is your actual transfer function $T(\omega)$ is your target (ideal) and $W(\omega)$ is a weighting function all sampled on a discrete frequency grid $\omega_k$ . For something like a lowpass a good initial choice for the weighting function is $$W(\omega) = \min(\frac{1}{|T(\omega)|^2},A_{max})$$. For a simple lowpass filter with a fixed stopband target of $A_{max}$, this simply becomes

$$ W({\omega_k}) = \begin{cases} 1, & \omega_k < \omega_c\\ A_{max}, & \omega_k > \omega_c \end{cases} $$

where $A_{max}$ is the maximum stop attenuation you actually need (in power units).

You can fine tune this metric by choosing a "representative" frequency grid $\omega_k$ and tweaking the weighting function. Your filter MUST have a transition band between start and stop band, if you don't care about the exact shape of the transition band, simply exclude it from the frequency grid or assign it a low weighting.

This specific metric is written as a difference of complex numbers, i.e. it combines magnitude and phase errors into the same metric. If you have different sensitivities to the errors, you can certainly split this into two separate metrics using the same method.

Time domain transients are harder to assess. They are implicitly included in the transfer function error phase metric, but this may or may not be representative of how your application reacts to it.

only linear phase FIRs were considered for now

Double edged sword: Yes, there is no phase distortion but you loose causality and there is pre-ringing, i.e. sharp onsets are smeared towards negative time, not just positive time.

A completely different metric (and a very good one) is "Application Performance Testing". For example: if you work on a digital communication system, create a set of representative test signals, run it through a full end-to-end model/prototype of your system, plug in different filters and see what happens to the bit error rate. Same process for a Natural Language Processing front end: apply different filters to representative test signals and measure the word detection rate of the voice engine.