Types of rounding in coefficients quantization

Question

Suppose we have create an IIR filter with matlab function "ellip", and then we want to quantize the coefficients using:

\begin{align*} bq=Quantize('round',b,2^8); \cr aq=Quantize('round',a,2^8); \end{align*}

I have read that there are 4 major types of rounding:

• truncate
• round
• convergent rounding
• round-to-zero

What is the differences between of them and how i know which method is best to choose?

Answer 1

The various rounding methods have a computation vs. quantization error tradeoff.

Truncate

Truncation is the simplest method. Everything after the decimal point is simply lopped off. For instance, both 2.1 and 2.9 become 2. This is very simple, but is the worst method in terms of quantization error. It is particularly bad because when you are dealing with non-negative numbers it introduces a strong negative bias. In some algorithms that can be bad.

Round/Convergent Rounding

Simply saying "Round" doesn't tell you enough to know what is meant. What kind of rounding? People usually mean convergent rounding when they say "round", so I will assume that that is what is meant.

Convergent rounding rounds down when the decimal place is $\le .499\overline{9}$ and rounds up when the decimal place is $\ge .500\overline{0}1$. The question remains- what to do when you have exactly $.5$? Some rounding algorithms always round down (this is called "round-to-zero"), but that introduces a very small amount of bias. Convergent rounding tries to eliminate the bias by rounding to the nearest even number, the assumption being that around half the time that will be up and half the time it will be down.

This is the best algorithm in terms of quantization noise, but is the most computationally intense. In many situations though, you don't care how computationally intense it is since the calculation isn't done at run-time.

Round-to-Zero

As mentioned previously, round-to-zero is the same as convergent rounding except it always rounds towards zero when the decimal portion is $.5$. This is a compromise between truncation (easy computationally) and convergent rounding (low quantization error).

Answer 2

Different types of rounding:

1. truncate: choose the nearest integer that's smaller than the actual number, also known as "round towards -infinity". That's then default for many compilers as it's the easiest to implement in hardware.
2. round: choose the nearest integer. That minimizes overall noise energy.
3. round-to-zero: chose the nearest integer that's smaller in magnitude. Example 1.8->1, -1.8->-1
4. Convergent rounding: very similar to "round", only the way exact half (2.5, -4.5 etc.) are different to avoid a rounding bias. Very little practical difference, so we ignore it

Typically truncate is the worst, but cheapest; round minimizes overall noise, and round-to-zero maximizes stability and minimizes limit cycles. However that only applies to signal conversion in, for example IIR Filters.

Coefficient quantization is more complicated. First you need to define what you care most about: Is it magnitude response, phase response, are there any "don't care" regions in frequency, etc. It's typically best to try all possible combinations and then pick the best one based on a pre-defined some error criteria. In extreme cases (poles very close to the unit circle), the quantization can kick a pole out of the unit circle and the filter becomes unstable.

The quantization also needs to take the filter topology into account. Typically this filters are broken down in biquads and the specific implementation of the biquad will impact the coefficient representation and the sensitivity to quantization.

If in doubt, try them all, that's what computers are good at.