5
$\begingroup$

The uncertainty principle is usually stated as a relationship between a continuous signal and that signal's Fourier transform, and says that $$ \int_{-\infty}^{\infty} \! x^2 f(x) \ \mathrm{dx} \int_{-\infty}^{\infty} \! \xi^2 \hat{f}(\xi) \ \mathrm{d\xi} \ge \frac{1}{16\pi^2}. $$

Apparently it is somewhat difficult to define an equivalent relationship for discrete time signals, but, for "reasonable" bandlimited signals with "reasonable" transforms, it is basically correct to say that the above holds true of the continuous signal reconstructed from the samples. For a good overview I found the following non-paywalled paper useful: Venkatesh, Kumar Raja, Vidyasagar: On the uncertainty inequality as applied to discrete signals, Int'l J. Math. and Math. Sci., 2006:48185, 2006. doi:10.1155/IJMMS/2006/48185.

So I also understand what this says about FIR filters: I can treat my kernel as a signal, apply the uncertainty principle and see that if I want a very narrowband filter, I'm going to have to have a lot of taps.

My question is what does the uncertainty principle say about an IIR filter? If I have a very narrow band IIR filter it would seem that the uncertainty principle would say that I would have to trade something off. What? I can't figure out what the appropriate signal is that I should take the variance of. Is it continuous reconstruction of the impulse response of my filter? (I.e., so I should expect a very narrow-band IIR filter to have an extremely long startup transient?)

$\endgroup$
  • 1
    $\begingroup$ It would be sort of strange if the UP says something particular for IIR that does not apply to FIR and vice versa. In this context the UP says something about impulse responses. You mention the example of narrow band IIR but consider narrow band FIR for very low frequency, will that not have long startup transient? Just as the narrowband FIR need many taps so does the IIR impulse response. I'm not really sure though. Just a thought. $\endgroup$ – niaren May 30 '13 at 8:07
3
$\begingroup$

The uncertainty principle basically says the same thing for IIR filters: "the steeper the filter, the longer the Impulse Response". The only difference between FIR and IIR is that you have to use a somewhat different definition of "filter length".

While all IIR filters are technically infinite, they have very different decay rates. You can define a quantity that basically describes "energy decay over time". For most IIR filters this is an exponential decay (often dominated by the pole-pair closest to the unit circle) and the decay rate is directly related to the filter bandwidth.

An interesting quantitative approach to determining decay rates can be found in the area of reverb time calculation, using "time reversed impulse response integration".

$\endgroup$
  • 1
    $\begingroup$ +1. I think one place where people sometimes get confused is in thinking that IIR filters have shorter impulse responses than FIR filters because you can accomplish the same filter specifications with a much smaller-order IIR filter. $\endgroup$ – Jason R May 30 '13 at 13:48
  • $\begingroup$ Yes, one would think that the "infinite" in IIR might be a bit of a give away, but, sadly, it's not. :-) $\endgroup$ – Peter K. May 30 '13 at 15:49
  • $\begingroup$ "the steeper the filter, the longer the Impulse Response". Do you think/claim that the UP explicitly or implicitly says that a brickwall filter has infinite impulse response? $\endgroup$ – niaren May 30 '13 at 16:36
  • $\begingroup$ Not sure whether the UP says this implicitly, but it's certainly true since the impulse response of a brickwall filter is a sin(x)/x function (potentially cascaded with some allpass) $\endgroup$ – Hilmar May 31 '13 at 10:41
2
$\begingroup$

Your Heisenberg-type uncertainty principle should read:

$$\left(\int_{-\infty}^\infty x^2|f(x)|^2\,dx\right)\left(\int_{-\infty}^\infty\xi^2|\hat{f}(\xi)|^2\,d\xi\right)\geq \frac{1}{16\pi^2}$$

Otherwise you could cheat with negative values. A stronger Heisenberg-type uncertainty principle takes into account the means of $|f(x)|^2$ and $|\hat{f}(\xi)|^2$, nice if you have either of them shifted away from zero like in a delay or a complex band-pass filter:

$$\left(\int_{-\infty}^\infty (x-x_0)^2|f(x)|^2\,dx\right)\left(\int_{-\infty}^\infty(\xi-\xi_0)^2|\hat{f}(\xi)|^2\,d\xi\right)\geq \frac{1}{16\pi^2}$$

This still has trouble with most real-valued $f(x)$, because for those $|\hat{f}(\xi)|^2$ is zero-mean while it may be symmetrically concentrated away from $\xi=0$. You may be tempted to include only $\xi>0$ in the integral of the uncertainty principle, but that makes it only approximately correct. Instead, an even stronger uncertainty principle, the Hirschman uncertainty (entropic uncertainty) can be used. Quoting from Wikipedia:

A stronger uncertainty principle is the Hirschman uncertainty principle, which is expressed as:

$$H(|f|^2)+H(|\hat{f}|^2)\ge \log(e/2)$$

where $H(p)$ is the differential entropy of the probability density function $p(x)$:

$$H(p) = -\int_{-\infty}^\infty p(x)\log(p(x)) \, dx$$

where the logarithms may be in any base that is consistent. The equality is attained for a Gaussian, as in the previous case.

I'm calling the left side of the first equation of the quote Hirschman uncertainty. For all of these uncertainty principles it is assumed that $f(x)$ and consequently its Fourier transform are normalized as:

$$\int_{-\infty}^\infty |f(x)|^2\,dx = 1 \Leftrightarrow \int_{-\infty}^\infty|\hat{f}(\xi)|^2\,d\xi=1$$

Hirschman uncertainty is difficult to explain, because it seems to penalize for values of $|f(x)|^2$ and $|\hat{f}(\xi)|^2$ near $1/e$, where $e$ is the base of the natural logarithm $\ln$ (but not necessarily the base of the logarithm used in the Hirschman uncertainty principle):

-p(x)\ln(p(x))
Figure 1. $-p(x)\ln(p(x))$ as function of $p(x).$

Hirschman uncertainty has no difficulties handling $p$ that have higher moments than mean and variance, which can make it more suitable for analysis of IIR filters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.