Basic Questions on Wiener Filtering

Question

I read a lot about Wiener Filters (focusing on discrete time case). I understand the math, but I am quite disconnected from the real life assumptions behind using such a filter. Unfortunately textbooks are just laying out the formulas without putting forward much intuition in it.

The task is to extract x(t) given y(t), where y(t) = x(t) + n(t). Great but first we need to build an appropriate filter. At this point:

1. Are we assuming that we have access to some training samples of x(t) and/or n(t) in addition to y(t)?

2. Or are we just assuming that we have access to statistical properties(auto correlation, cross correlation) of x(t) and/or n(t) in addition to y(t)? If so since x(t),n(t) are not explicitly available, are these statistics computed from other signals available to us with similar statistical properties?

3. Or is it a combination of both, such as having access to some training samples of x(t), but having only access to second order statistics of n(t) (or vice versa)?

4. Is the noise n(t) always assumed to be uncorrelated to x(t) in Wiener filtering, or are the textbooks going for the simplified version? What is the default scenario?

5. Why are all the examples coming up with all-zero filters but not using pole-zero filters? Is it just a matter of simplified textbook treatment or is the problem definition not amenable to using transfer function modelling approach? Yes the first approach is computationally easier, but these days we have significant computing power and I don't think it is a good justification anymore, if that's the only reason.

Looking forward to hear from practitioners.

Answer 1

The task is to filter x(t) when given y(t), where y(t) = x(t) + n(t). Great but first we need to build an appropriate filter. At this point:

No. The task is to filter $y[n]$ to achieve $x[n]$. Your filter acts on the available signal $y[n]$ to get a best estimate of $x[n]$ from a noisy observation of it. BTW. discrete-time uses $y[n]$ or at least $y(n)$ notation, and not $y(t)$. In DSP, $t$ is reserved for continuous-time.

1) Are we assuming that we have access to some training samples of x(t) and/or n(t) in addition to y(t)?

trainig samples is a term related with adaptivity, pattern recognition, neural networks etc. In the classical Wiener context, you do not need training samples but you need data statistics. See answer 2 for how to obtain the statistics.

2) Or are we just assuming that we have access to statistical properties(auto correlation, cross correlation) of x(t) and/or n(t) in addition to y(t)? If so since x(t),n(t) are not explicitly available, are these statistics computed from other signals available to us with similar statistical properties?

Yes. In the Wiener filter context, you either assume (for theoretical development) or estimate (for practical applcations) the correlations (or Power Spectral Densities) between all those signals. In practice you estimate them from avaiable data by several means, depending on the application. You need data for this; and that's probably what you call training samples.

3) Or is it a combination of both, such as having access to samples of x(t), but having only access to second order statistics of n(t) (or vice versa)?

You do not have access to relevant noise-free $x[n]$ by any means; otherwise why filter $y[n]$ to get $x[n]$? But, in some cases, one can have noise free irrelevant blocks of $x[n]$ samples, in addition to the relevant noisy blocks. So you can use those noise free blocks to estimate certain parameters of the process $x[n]$, so as to filter out the noisy blocks.

4) Is the noise n(t) always assumed to be uncorrelated to x(t) in Wiener filtering, or are the textbooks going for the simplified version? What is the default scenario?

Yes. The noise $n(t)$ (or better $v[n]$ in discrete-time context) is almost always modeled as additive, white, zero-mean Gaussian noise uncorrelated with $x[n]$. This simplifies all the mathematical development and in typical cases is a good model. But you can relax some of those assumption, if that would yield a better (but more complex to derive) filter...

5) Why are all the examples coming up with all-zero filters but not using pole-zero filters? Is it just a matter of simplified textbook treatment or is the problem definition not amenable to using transfer function modelling approach? Yes the first approach is computationally easier, but these days we have significant computing power and I don't think it is a good justification anymore, if that's the only reason.

IIR filters are good for their computational efficiency (and in Wiener context, they also yield the minimum MSE compared to FIR) but they may suffer from unstability and causality problems, and their design and analysis is more complex in adaptive filtering context. For this reasons, it's simpler and safer to use FIR based filters for adaptive Wiener applications.

To learn more about the Wiener filters and adaptive Wiener filtering:

• Simon Haykin - Adaptive Filter Theory.

• Monson Hayes - Statistical Digital Signal Processing and Modeling.

Answer 2

In some applications, such as channel equalization, one often uses a training sequence known to the receiver to compute the optimum filter coefficients. During normal operation, the training sequence is replaced by the decisions of the receiver ("decision-directed mode").

In other applications, such as speech denoising, one attempts to estimate the properties of the noise during silence periods. Of course it's crucial to detect such periods of signal absence with great accuracy. The power spectrum of the desired signal can then be estimated by subtracting the estimated noise spectrum from the spectrum of the observed noisy signal (assuming that signal and noise are uncorrelated).

Wiener filtering does not generally assume that signal and noise are uncorrelated. E.g., the non-causal Wiener filter is given by the well-known formula

$$H(z)=\frac{S_{xy}(z)}{S_{yy}(z)}\tag{1}$$

where $S_{xy}(z)$ is the cross-power spectral density of the signal and the noisy observation, and $S_{yy}(z)$ is the power spectrum of the observation.

Only if noise and desired signal are uncorrelated do we get the intuitively even more pleasing formula

$$H(z)=\frac{S_{xx}(z)}{S_{xx}(z)+S_{nn}(z)}\tag{2}$$

Note that in many practical situations it is reasonable to assume that desired signal and noise are uncorrelated, but the Wiener filter does not assume this a priori.

Many textbooks discuss three cases of the Wiener filter: 1. the non-causal Wiener filter, 2. the causal solution, which is generally IIR, and 3. the (causal) FIR solution. The latter is often emphasized because of its straightforward implementation and the availability of stable adaptive algorithms. Many textbooks also give examples of the IIR solution. One such book that I can recommend is Optimum Signal Processing by S.J. Orfanidis.