The article Efficient Use Of Sparse Adaptive Filters (Proc. Asilomar Conference, Khong et al., 2006) introduces adaptive filters for the estimation of channels or systems having a sparse impulse response. Assuming that the channel is modeled by an FIR model, then the output of the channel will be given by $$y[n] = \sum_{i=0}^{p-1} h[i] x[n-i] \tag{1}$$ where $x$ is the input to the model and $h$ is the MA coefficients. As an example, consider an MA(3) model of order p=3 having coefficients h = [1,0.2,0.5,0.6]

I can expand the equation to write as $y[n] = x[n] + 0.2*x[n-1] + 0.5*x[n-2] + 0.6*x[n-3]$

My problems are:

  1. From a time series or statistics point of view, FIR filter = Moving Average and IIR = Autoregressive model. I cannot understand how to relate the delays, 1,2,3 denoted by the order to the lags terminology often used in statistics. The impulse are known as the coefficients of the model and how to relate the impulse to the coefficients?
  2. Why or rather what is the intuitive meaning of the coefficients becoming zero?
  3. When does that happen/some real life examples of applications?

2 Answers 2


The notion of sparsity entails that an object, living for instance in an $n$-dimensional space, can be described (in the suitable basis/frame) by a number $k$ of meaningful components (each above a threshold, or whose combination is close enough to the signal) "much smaller" than $n$.

When talking about filter identification (adaptive or not), the filter is, at the beginning, unknown. To avoid looking inside an infinity of possibilities, one first restricts the space of solutions to the subspace of length-$n$ or order-$n$ filters. Either for true physical reasons, out of wishful thinking or heuristic motivations, or because the optimisation problem is not directly tractable (non-smooth, non-convex), one may call for Ockham's razor, or the law of parsimony, one may want to restrict the solution (not the real object) to be $k$-sparse.

Your example could be sparse per se, if initially the space of solutions could have up to 100 taps. Additionally, it could perhaps be approximated (in practice) by a sparser filter: $$y[n] = x[n] + 0.6*x[n-3]$$ discarding the two smallest coefficients.

With this introduction, let us go back to your questions:

  1. I would disagree with "FIR filter = Moving Average and IIR = Autoregressive model": an IIR can be AR/MA, some FIR can be rewritten in an AR fashion. The rest of the question is unclear to me: if the model is linear, the impulse response would be some sparse approximation of the actual coefficients.
  2. The actual coefficients are not becoming zero. But, either they form a filter short enough to be qualified as sparse, or their estimate has very few non-zero taps
  3. It may happen in some echo cancellation problems: either in a room or on delay lines, one expect that the source bounces after some travelling back and forth. If at each bouncing the reflection is scarce, and there is some attenuation, the "global" echo filter could look like (two bounces) $$y[n] = x[n] - 0.25*x[n-99]+ 0.5*x[n-100]+ 0.25*x[n-101] + 0.125*x[n-199]+ 0.25*x[n-200]+ 0.125*x[n-201]$$ hence a sparse filter, even with order 201. Outside speech processing, we used such priors in seismic processing of remove multiples, approximate models of bouncing waves, assuming that the filters were sparse at boundaries between rock layers, see A primal-dual proximal algorithm for sparse template-based adaptive filtering: Application to seismic multiple removal, 2014, IEEE Transactions on Signal Processing:

Unveiling meaningful geophysical information from seismic data requires to deal with both random and structured “noises”. As their amplitude may be greater than signals of interest (primaries), additional prior information is especially important in performing efficient signal separation. We address here the problem of multiple reflections, caused by wave-field bouncing between layers. Since only approximate models of these phenomena are available, we propose a flexible framework for time-varying adaptive filtering of seismic signals, using sparse representations, based on inaccurate templates. We recast the joint estimation of adaptive filters and primaries in a new convex variational formulation. This approach allows us to incorporate plausible knowledge about noise statistics, data sparsity and slow filter variation in parsimony-promoting wavelet frames. The designed primal-dual algorithm solves a constrained minimization problem that alleviates standard regularization issues in finding hyperparameters. The approach demonstrates significantly good performance in low signal-to-noise ratio conditions, both for simulated and real field seismic data.

Note that parsimony, usually resorting to an $\ell_0$ count measure, is not easy to optimize, as non-differentiable, non-convex, etc.


You can view each x(time instant) as a sample. For every new sample into a system you will have a new output. The coefficients are effectively zero for every sample that is older than the 4 most recent samples.

All physically implementable (real) systems have a constraint on the number of coefficients, otherwise an infinite number of taps (i.e. for a FIR filter) would be necessary.

If you have an impulse into your system, on the 5th "time instant" the impulse response would be zero. If your first sample value was 1 and the second was -0.2, your output would also be zero.

  • $\begingroup$ Hi: This is where econometrics really differs from DSP. If you assume that the impulse response is equal to 1 ( i.e: e_o = 1 ) then you have an impulse response which, for an MA model is just the moving sequence of the ma coefficents in an MA model. ( for an AR, it's different. see distributed lag models for the impulse response in that case ). But the difference between here and DSP is that there are stochastic coefficients entering at each time step ( the e_i) so you can't use convolution to arrive at the response. See Box and Jenkins text for forecasting approaches with MA and AR models.. $\endgroup$
    – mark leeds
    Commented Dec 13, 2017 at 19:49
  • $\begingroup$ should have said "inputs" above rather than "coefficients". but I can't edit ? $\endgroup$
    – mark leeds
    Commented Dec 13, 2017 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.