# How to create a distortion compensation filter

I am creating a channelizer that consists of a complex mixer, CIC decimator, and a FIR compensation/decimation filter. The final FIR filter may, if it matters, be implemented as more than one filter.

My question is, how do I design a filter such that it compensates for the very non-flat frequency response of the CIC filter? Do you create the frequency response that you want by calculating the reciprocal of the CIC's response and then run it through an inverse FFT to get the impulse response?

As you can see, though my particular problem pertains to CIC filters, my question is really about how you create any kind of distortion compensation filter.

• A filter whose purpose is to compensate for distortion earlier in the communication system is often called an equalizer; that may give you some more information to read about in the meantime. Common types are the zero-forcing equalizer (which is not what you want if the system you're compensating for has zeros in its frequency response) and the minimum mean-squared error (MMSE) equalizer. – Jason R Feb 22 '12 at 23:10
• @JasonR I'm familiar with MMSE Adaptive Equalizers in the context of channel compensating decision-directed equalizers, but I doubt that that's what you meant. Would you train the equalizer somehow? I guess the crux is how you determine what the error is. – Jim Clay Feb 23 '12 at 1:30
• MMSE only refers to the criterion that the equalizer strives for; it can be adaptive or non-adaptive. If you know the transfer function of the system to compensate, and the autocorrelation function of any additive noise, and both are time-invariant, then you can derive "the" MMSE equalizer, which will be fixed over time. Adaptive solutions can be used when either that information is not known precisely or it changes over time (which is pretty common). – Jason R Feb 23 '12 at 2:46
• Adaptive filters often include a training mode where a number of known symbols are transmitted through the system, allowing the receiver to let its filter converge on a good set of coefficients. After training, it's also common to enter a tracking phase, where decision-directed operation is used for continual updates of the filter taps. As an alternative, in a regime where SNR and ISI is "good enough," decision-directed mode could be used from the beginning as a blind acquisition scheme, skipping the training phase. – Jason R Feb 23 '12 at 2:48
• Finding reading material on this subject that is mathematically accessible is somewhat difficult. I like Haykin's Adaptive Filter Theory, but it takes some effort to cut through the math (and you need some solid linear algebra background). It's complicated enough that if you don't use it regularly, you'll need to go and brush up on statistical signal processing topics first. – Jason R Feb 23 '12 at 2:51

The following addresses the CIC compensation and is not a general "distortion" technique. But it is a straight-forward method to "compensate" non-varying "distortion". If the frequency response is known the inverse of the frequency can be used to compensate. Examples like the CIC filter, where a poor filter might be used because of reduced complexity, are compensated later the signal chain. In this example the frequency response is know and the inverse can be used. Note, with multi-rate filters you only want to use the "usable" spectrum after decimation.

Generally, to compensate a CIC filter the inverse of the CIC filters response can be used to generate the compensation filter. The CIC has a response of (see reference [r2] or [r3])

$$H(\omega) = \left| \frac{sin(\omega D/2)}{sin(\omega M/2)} \right|^N$$

Where D is the differentiate delay, M is the decimation rate, and N is the filter order (number of cascaded filters). The inverse can be specified as

$$H(\omega) = \left| \frac{sin(\omega M/2)}{sin(\omega D/2)} \right|^N$$

Once we have the frequency response of the compensation filter, we can simply choose the length of FIR filter that we desire. The length of the FIR is application specific. Obviously the longer the FIR filter the better compensation.

The following are plots of this straight forward compensation.  The following is the Python code to create the frequency responses and plots.

import numpy as np
from numpy import sin, abs, pi
import pylab

D = 1; M = 7; N = 3

Hfunc = lambda w : abs( (sin((w*M)/2)) / (sin((w*D)/2.)) )**N
HfuncC = lambda w : abs( (sin((w*D)/2.)) / (sin((w*M)/2.)) )**N

w = np.arange(1024) * pi/1024

G = (M*D)**N
H = np.array(map(Hfunc, w))
Hc = np.array(map(HfuncC, w))
# only use the inverse (compensation) roughly to the first null.
Hc[int(1024*pi/M/2):] = 1e-8
plot(w, 20*log10(H/G))
plot(w, 20*log10(Hc*G))
grid('on')


See [r1] for other approaches and $sinc^{-1}$ approximation.

[r2] R. Lyons, "Understanding Digital Signal Processing", 2nd ed., Prentice Hall, Upper Saddle River, New Jersey, 2004

[r3] R. Lyons, "Understanding Cascaded Integrator Comb Filters"

• Ref 1 is great, but uses different terminology (M = stage delay, where you used D. R = Reduction rate, where you used M) so it took me a while to see the error: you use WM/2, Altera specifies pi F /R. After a quick Excel plot, I believe the Altera version is the correct one. – Alan Campbell Aug 4 '15 at 5:16
• @AlanCampbell If you plot the above equations (which I did in the code snippet) you will see there is no error. What I didn't explicitly call out in the equations is the gain. Notice in the code snip I remove the gain. If you run the code snip and modify the M=8 and N=9 it creates the same plots as reference 1. I used the nomenclature closer to reference 2. – Christopher Felton Dec 9 '15 at 19:12

I think Christopher's answer is a good one. I thought that I would add one for us lazy/cheater types.

While digging around Matlab's fdatool (Filter Design & Analysis Tool) I discovered that it can design and model both CIC and inverse sinc filters, where the inverse sinc filter is the CIC's distortion compensation filter.

You generate the CIC filter by going straight to the "Create a multi-rate filter" page (button on the left in version R2011b) and specifying the CIC filter. You can then set all the parameters that you want on it, such as interpolation/decimation rates, number of cascades (Matlab calls it "sections"), etc.

You create the inverse sinc filter by going to the "Design filter" page (the page that the tool defaults to when it starts), and specifying "Inverse Sinc Lowpass" from the Lowpass drop down menu. You can then set the various parameters for it. It has a couple of unique parameters in the Options part of the dialog, including "p", which corresponds to the number of cascades (sinc exponent).

The inverse sinc filter design was not completely satisfying, though, since getting it right seemed less an instance of doing the math than figuring out where my pass band was, what the CIC droop was at that point, and then designing a filter that roughly canceled out that droop. Since my CIC filter had four cascades, I would have thought that I would need to set "p" to four, but it way overcompensated when I did that. I ended up leaving "p" at 1, the default value.

The tool really shone when I saved both filters in my session, and then went into the filter manager and cascaded them to see the overall frequency response. This was very helpful, and easy to do.

Without duplicating the response here, please see

how to make CIC compensation filter

which shows a very simple 3 tap solution for CIC compensation.

This is specific to CIC or any application for an inverse Sinc is required (such as prior to D/A conversion).

To compensate for any general transversal distortion (that which could be replicated with a transversal filter as opposed to dynamic noise distortion) I would take the approach of using the Wiener Hopf equations to come up with a compensation filter using the Least Squared Error approach, ideally using a training sequence if available to compare desired to actual response and train the equalizer. For an application showing this with source code used see:

Compensating Loudspeaker frequency response in an audio signal