# What is an “innovation filter”?

I'm a math postgrad student working through a paper on eigenvalue decompositions of matrices of FIR filters (used for stuff like total decorrelation, convolutive mixing, MIMO). Towards the beginning, when describing their signal model, they use the phrase:

If the PSD of the $$l^{th}$$ source is generated by a stable and causal innovation filter $$\text{F}_{l}(z)$$, ...

What does this mean? I understand a stable filter is one whose coefficients $$a[n]$$ approach 0 as $$n \rightarrow \infty$$, and a causal filter is one which does not use values from the future (i.e. no negative time lags). I have read that the innovation in terms of stochastic processes is defined as the difference between the actual next value of a time series, and the value given by an optimal prediction based on the information already available.

I cannot, however, find any description of an innovation filter. I have a mathematics background rather than a DSP/engineering background, and so sometimes some of the terminology trips me up, which is why I was hoping DSP stack exhange could kindly help!

• The term innovations filter is usually encountered in Kalman Filtering, see Kailath, Thomas. "A view of three decades of linear filtering theory." IEEE Transactions on Information Theory 20.2 (1974): 146-181. – Stanley Pawlukiewicz Sep 21 '18 at 1:33
• Hi: Stanley's reference is useful but more general than this one which focuses specifically on the innovations filter. metronu.ulb.ac.be/npauly/art_2014_2015/kailath_1968.pdf – mark leeds Sep 21 '18 at 5:23
• Thank you both, I will have a look at these references shortly. – A. White Sep 22 '18 at 2:18

An innovations filter of a WSS process $$X(t)$$ is a causal and stable minimum phase filter that can be used to generate $$X(t)$$ from a white noise input $$N(t)$$:

$$X(t)=\int_0^{\infty}h(\tau)N(t-\tau)d\tau\tag{1}$$

where $$h(t)$$ is the impulse response of the innovations filter.

The (causal and stable) inverse filter of $$h(t)$$ is called the whitening filter of $$X(t)$$. Its response to the input $$X(t)$$ is white noise $$N(t)$$.

A process that can be represented by $$(1)$$ is called a regular process. Note that its power spectrum is given by

$$S(\omega)=|H(\omega)|^2\tag{2}$$

where $$H(\omega)$$ is the Fourier transform of the impulse response $$h(t)$$ of the innovations filter (assuming that $$E\{N(t)N(t+\tau\}=c\delta(\tau)$$ with $$c=1$$). The requirement of causality of the innovations filter implies that the power spectrum $$S(\omega)$$ of $$X(t)$$ satisfies the Paley-Wiener condition

$$\int_{-\infty}^{\infty}\frac{|\log S(\omega)|}{1+\omega^2}d\omega\lt\infty\tag{3}$$

This implies that $$S(\omega)$$ cannot contain spectral lines, and it cannot be band-limited. Unlike a singular process (consisting of spectral lines), a regular process cannot be parameterized by a finite set of random variables and it is not predictable, i.e., it is not completely determined in terms of its past.

Reference: Probability, Random Variables, and Stochastic Processes, A.Papoulis, Athanasios 1965. McGraw-Hill.

• Great, this exactly what I needed to know, thank you very much. What implications for a signal does it have that its PSD is generated by such a filter? – A. White Sep 22 '18 at 2:17