# ensure stability against coefficient quantization

for a filter:

$$H(z)=\frac{1+0.1z^{-1}}{1+0.1z^{-1}+0.9998z^{-2}}$$

Which precautions could be taken to ensure that the filter does not become unstable because of coefficient quantization? I've think that it would depend of the implementation chosen (I've read there are implementations pretty robust against quantization), but not really sure. Thanks in advance.

your filter is in the form of a standard biquad with 5 independent coefficients ($b_2=0$ in your case):

$$H(z) = \frac{b_0 + b_1 z^{-1} + b_2 z^{-2}}{ 1 + a_1 z^{-1} + a_2 z^{-2}}$$

your two poles are the value of $z$ that make the denominator equal to zero

$$1 + a_1 z_p^{-1} + a_2 z_p^{-2} = 0$$

or (multiplying both sides by $z_p^2$),

$$z_p^2 + a_1 z_p + a_2 = 0 \tag{1}$$

and stability depends entirely on whether your poles (after coefficient quantization) are contained in the unit circle:

$$|z_p| < 1$$

or

$$|z_p|^2 < 1$$

the solution to Eq. 1 is (quadratic formula):

$$z_p = -\frac{a_1}{2} \pm \sqrt{\left(\frac{a_1}{2}\right)^2 - a_2}$$

and, if the poles are complex conjugate, then $\left(\frac{a_1}{2}\right)^2 < a_2$ and your complex conjugate poles are

$$z_p = -\frac{a_1}{2} \pm j \sqrt{a_2 - \left(\frac{a_1}{2}\right)^2}$$

in the case of two real poles, when $\left(\frac{a_1}{2}\right)^2 \ge a_2$, you must guarantee (to ensure stability):

$$-1 < -\frac{a_1}{2} \pm \sqrt{\left(\frac{a_1}{2}\right)^2 - a_2} < +1$$

that means both of the following must be guaranteed (after quantization):

$$-\frac{a_1}{2} + \sqrt{\left(\frac{a_1}{2}\right)^2 - a_2} < +1$$

$$-1 < -\frac{a_1}{2} - \sqrt{\left(\frac{a_1}{2}\right)^2 - a_2}$$

what is necessary (but not sufficient) is that $|a_1| < 2$ and what also must be true is (getting rid of the $\sqrt{\cdot}$)

$$\left(\frac{a_1}{2}\right)^2 - a_2 < \left(1 + \frac{a_1}{2}\right)^2 = 1 + a_1 + \left(\frac{a_1}{2}\right)^2$$

and

$$\left(\frac{a_1}{2}\right)^2 - a_2 < \left(1 - \frac{a_1}{2}\right)^2 = 1 - a_1 + \left(\frac{a_1}{2}\right)^2$$

or

$$a_2 > -1 - a_1$$

$$a_2 > -1 + a_1$$

so for real poles ($\left(\frac{a_1}{2}\right)^2 \ge a_2$) you must make sure, after quantization that

$$|a_1| < 2$$ and $$-1 + |a_1| < a_2 \le \left(\frac{a_1}{2}\right)^2$$

for complex conjugate poles ($\left(\frac{a_1}{2}\right)^2 < a_2$), it's much, much easier. i'll leave this for an exercise for the reader but you can easily show that (for complex conjugate):

$$|z_p|^2 = a_2$$

which means

$$0 < |z_p|^2 = a_2 < 1$$

and you still will see that $|a_1| < 2$ is necessary because $|a_1| < 2 a_2 < 2$. it turns out that $-\frac{a_1}{2}$ is the point on the $\mathfrak{Re}(z)$ axis that is midway between the two poles, whether they be real or complex conjugate.

now, if your digital filter is slewing from one stable complex pole case to another stable complex pole place, then $a_2$ is slewing from one value less than 1 to another value less than one and all filters in between will have $a_2$ less than one and will be stable, at least in a "static" sense. rapid coefficient slewing can create instabilities in and of itself, but if the slewing stops (at a stable condition), any instability also stops.

so in both real and complex conjugate cases, to have stability in the above biquad you must have

$$-2 < a_1 < 2$$

and

$$0 < a_2 < 1$$

for the complex conjugate case $a_1^2 < 4 a_2$

or

$$-1 + |a_1| < a_2 \le \left(\frac{a_1}{2}\right)^2 = \frac{|a_1|^2}{4}$$

for the real pair case $a_1^2 \ge 4 a_2$.

note that, in the real pole pair case, we know that $$-1 + |a_1| < \frac{|a_1|^2}{4}$$ because $$-1 + |a_1| - \frac{|a_1|^2}{4} = -\frac{1}{4}(|a_1|-2)^2 < 0$$. so there is always a gap for $a_2$ to fit into.

• perhaps a more succinct way to put is is, first choose any $a_1$ so that $$-2 < a_1 < 2$$ then, for a stable filter, choose $a_2$ so that $$-1 +|a_1| < a_2 < 1$$ if $$-1 +|a_1| < a_2 \le \frac{a_1^2}{4}$$ it's a stable filter with real poles ("overdamped" or "critically damped"). if $$\frac{a_1^2}{4} < a_2 < 1$$ it's a stable filter with complex conjugate poles ("underdamped" with an impulse response having a resonant sinusoid in it). – robert bristow-johnson Jun 23 '15 at 1:29

The poles of that filter are extremely close to the unit circle (pole radius $=0.9999$). You will run into problems with any fixed-point implementation of that filter (with any reasonable word length), because it is not only coefficient quantization that will give you problems but also round-off noise (due to signal quantization). If you only consider coefficient quantization (not signal quantization), you need at least 14 bits for theoretical stability (for a direct-form structure). There are other filter structures, but they won't help much for such a critical case as given in your example.

What you need to do is ensure a certain stability margin, i.e. a maximum pole radius $r\le r_0$, where $r_0<1$. The appropriate value for the limit $r_0$ on the maximum radius depends on the available coefficient word length and on the chosen filter structure. Note that the shift of the pole locations due to coefficient quantization also depends on the pole angle (unless you use a coupled-form structure), so unfortunately there is no general rule for determining $r_0$.

EDIT:

I'll add some explanation in reaction to RBJ's comments. RBJ correctly claims that even when the coefficients are quantized, then, regardless of the number of bits, a second order section can always be made stable. This is obviously true, because we can always make sure that the (direct-form) coefficients $a_1$ and $a_2$ of the denominator polynomial $A(z)=1+a_1z^{-1}+a_2z^{-2}$ are inside the stability triangle:

\begin{align}a_2&\ge -a_1-1\\ a_2&\ge a_1-1\\ a_2&\le 1\end{align}

What I was talking about was the straightforward (or naive) quantization, where you just choose the quantized coefficient that is closest to the original coefficient given the number of available bits and the desired range of values. If you do this for the filter given in your question then you indeed need 14 bits in order to obtain a (theoretically) stable filter. As mentioned before, due to signal quantization that filter will still not be useful in practice.

So the approach I was talking about is to change the (unquantized) pole locations such that the maximum pole radius is less or equal to some chosen $r_0<1$, and then apply (naive) quantization (and of course check the result). RBJ's equivalent approach would be to do this in one step, i.e. quantize but at the same time guarantee that the quantized coefficients end up inside the stability triangle. Either way, you end up with a stable filter that deviates from the ideal filter. The magnitude of that deviation depends on how much the actual pole locations have to be changed in order to obtain a stable filter for the given number of bits spent on the coefficients.

• most of what Matt has written, i essentially dispute. we are both DSPers of experience, but my experience is evidently different than Matt's. for the complex-conjugate case, as long as the quantized $0<a_2<1$, in whatever precision, your filter will be stable. for the case of real poles, you have an additional hoop to jump through. and in both cases $|a_1|<2$, but that's a necessary, not sufficient requirement. – robert bristow-johnson Jun 22 '15 at 19:11
• also, just want to be clear: non-zero limit cycles are not the same as instability. yes, in fixed-point with poles very close to the unit circle, you can get limit cycles that result in your output stuck on a non-zero value long (like forever) after the input went to a dead zero. but that is not the same as instability which is violating BIBO. instability is still a mostly "linear" phenomenon, but limit cycles require something non-linear going on in there. – robert bristow-johnson Jun 22 '15 at 19:25
• Thanks very much for your answers, I'm just starting to get the concepts in DSP and my knowledge comes all from the theory rather than practice. I get the BIBO stability concept, so the idea is to keep always the quantized $a_2<1$. What I don't get too much is how can you do it. Is it a matter of implementation? I mean, how can I ensure this codition will hold after quantization? – Wobbler28 Jun 22 '15 at 19:42
• well, your theoretical $a_2$ should always be less than 1 if your theoretical biquad filter is stable. so if you always round down (or "round toward zero") in the quantization operation, your practical $a_2$ should be no closer to 1 than your original theoretical $a_2$. – robert bristow-johnson Jun 23 '15 at 0:48
• @robertbristow-johnson: I think we don't really disagree, we just see the same world through different eyes. I've added some text to my answer addressing your comments. – Matt L. Jun 23 '15 at 11:03

To answer the direct question: the easiest test is the following:

1. Quantize the coefficients
2. Turn the resulting fixed point numbers back into double precision floating point (i.e. something with significantly higher precision than your fixed point representation)
3. Re-Calculate the new pole locations and check whether they are still nicely inside the unit circle

This will also you allow to assess whether the quantization has changed the shape of the transfer function in an unacceptable way, even if it's still stable.

However, this specific filter will be very difficult to implement, it may be stable, but it will incredibly noise. The quantization during calculation induces the noise and this noise is amplified by the pole transfer function. This particular example has a maximum noise gain of 72 dB, so you will need a LOT of bits depending on what our signal to noise requirements are or consider implementing the fixed point math in double precision.

• "Turn the resulting fixed point numbers back into floating point.", Hil, are you sure that this will never change their value? it won't for 24-bit fixed (to IEEE single-precision float), but it may for 32-bit fixed. – robert bristow-johnson Jun 22 '15 at 20:43
• "This will also you allow to assess whether the quantization has changed the shape of the transfer function in an unacceptable way, even if it's still stable." from another answer, i would evaluate the frequency response from whatever coefficients (quantized or not) using $$20 \log_{10}|H(e^{j\omega})| \ = \ 10 \log_{10}\left( \left(\tfrac{b_0+b_1+b_2}{2}\right)^2 - \phi [4b_0b_2(1-\phi) + b_1(b_0+b_2)] \right) - 10 \log_{10}\left(\left(\tfrac{1+a_1+a_2}{2}\right)^2 - \phi [4a_2(1-\phi) + a_1(1+a_2)] \right)$$ where $\phi \triangleq \sin^2 \left( \frac{\omega}{2} \right)$ – robert bristow-johnson Jun 22 '15 at 20:51
• Fair about the type of floating point to be used and I edited my answer accordingly. – Hilmar Jun 22 '15 at 22:01
• A good method to analyze the effect of coefficient quantization is to design in a high precision domain, quantize, turn back into the high precision domain and look at the differences in the transfer function. "High precision" here means significantly higher than the fixed point domain. The key here is to do the comparison of before and after in the high precision domain – Hilmar Jun 22 '15 at 22:04