What does the uncertainty principle say about recursive filters?

Question

The uncertainty principle is usually stated as a relationship between a continuous signal and that signal's Fourier transform, and says that $$ \int_{-\infty}^{\infty} \! x^2 f(x) \ \mathrm{dx} \int_{-\infty}^{\infty} \! \xi^2 \hat{f}(\xi) \ \mathrm{d\xi} \ge \frac{1}{16\pi^2}. $$

Apparently it is somewhat difficult to define an equivalent relationship for discrete time signals, but, for "reasonable" bandlimited signals with "reasonable" transforms, it is basically correct to say that the above holds true of the continuous signal reconstructed from the samples. For a good overview I found the following non-paywalled paper useful: Venkatesh, Kumar Raja, Vidyasagar: On the uncertainty inequality as applied to discrete signals, Int'l J. Math. and Math. Sci., 2006:48185, 2006. doi:10.1155/IJMMS/2006/48185.

So I also understand what this says about FIR filters: I can treat my kernel as a signal, apply the uncertainty principle and see that if I want a very narrowband filter, I'm going to have to have a lot of taps.

My question is what does the uncertainty principle say about an IIR filter? If I have a very narrow band IIR filter it would seem that the uncertainty principle would say that I would have to trade something off. What? I can't figure out what the appropriate signal is that I should take the variance of. Is it continuous reconstruction of the impulse response of my filter? (I.e., so I should expect a very narrow-band IIR filter to have an extremely long startup transient?)

Answer 1

The uncertainty principle basically says the same thing for IIR filters: "the steeper the filter, the longer the Impulse Response". The only difference between FIR and IIR is that you have to use a somewhat different definition of "filter length".

While all IIR filters are technically infinite, they have very different decay rates. You can define a quantity that basically describes "energy decay over time". For most IIR filters this is an exponential decay (often dominated by the pole-pair closest to the unit circle) and the decay rate is directly related to the filter bandwidth.

An interesting quantitative approach to determining decay rates can be found in the area of reverb time calculation, using "time reversed impulse response integration".

Answer 2

Your Heisenberg-type uncertainty principle should read:

$$\left(\int_{-\infty}^\infty x^2|f(x)|^2\,dx\right)\left(\int_{-\infty}^\infty\xi^2|\hat{f}(\xi)|^2\,d\xi\right)\geq \frac{1}{16\pi^2}$$

Otherwise you could cheat with negative values. A stronger Heisenberg-type uncertainty principle takes into account the means of $|f(x)|^2$ and $|\hat{f}(\xi)|^2$, nice if you have either of them shifted away from zero like in a delay or a complex band-pass filter:

$$\left(\int_{-\infty}^\infty (x-x_0)^2|f(x)|^2\,dx\right)\left(\int_{-\infty}^\infty(\xi-\xi_0)^2|\hat{f}(\xi)|^2\,d\xi\right)\geq \frac{1}{16\pi^2}$$

This still has trouble with most real-valued $f(x)$, because for those $|\hat{f}(\xi)|^2$ is zero-mean while it may be symmetrically concentrated away from $\xi=0$. You may be tempted to include only $\xi>0$ in the integral of the uncertainty principle, but that makes it only approximately correct. Instead, an even stronger uncertainty principle, the Hirschman uncertainty (entropic uncertainty) can be used. Quoting from Wikipedia:

A stronger uncertainty principle is the Hirschman uncertainty principle, which is expressed as:

$$H(|f|^2)+H(|\hat{f}|^2)\ge \log(e/2)$$

where $H(p)$ is the differential entropy of the probability density function $p(x)$:

$$H(p) = -\int_{-\infty}^\infty p(x)\log(p(x)) \, dx$$

where the logarithms may be in any base that is consistent. The equality is attained for a Gaussian, as in the previous case.

I'm calling the left side of the first equation of the quote Hirschman uncertainty. For all of these uncertainty principles it is assumed that $f(x)$ and consequently its Fourier transform are normalized as:

$$\int_{-\infty}^\infty |f(x)|^2\,dx = 1 \Leftrightarrow \int_{-\infty}^\infty|\hat{f}(\xi)|^2\,d\xi=1$$

Hirschman uncertainty is difficult to explain, because it seems to penalize for values of $|f(x)|^2$ and $|\hat{f}(\xi)|^2$ near $1/e$, where $e$ is the base of the natural logarithm $\ln$ (but not necessarily the base of the logarithm used in the Hirschman uncertainty principle):

Figure 1. $-p(x)\ln(p(x))$ as function of $p(x).$

Hirschman uncertainty has no difficulties handling $p$ that have higher moments than mean and variance, which can make it more suitable for analysis of IIR filters.