Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.
Sign up to join this community
I have read that a Wiener filter is a filter used to produce an estimate of a desired or target random process by linear time-invariant (LTI) filtering of an observed noisy process.
Now, my doubt is regarding the use of the term LTI. How does the Wiener filter ensure that the filtering is linear time invariant?
An intuitive explanation would really help.
On of the easiest ways to see why the outcome of the problem that Wiener Filter tries to solve is Linear Time Invariant (LTI) Filter is to derive (Solve) it.
By the very definition of the Wiener filter we're after a solution in the form of a filter applied using convolution which means this filter must be LTI.
Let's go through this.
The Wiener Deconvolution problem is given by:
$$ y \left( t \right) = \left( h \ast x \right) \left( t \right) + n \left( t \right) $$
Where:
The Wiener deconvolution, in order to solve the above problem (Estimating $ x \left( t \right) $) is finding $ g \left( t \right) $ such that:
$$ \hat{x} \left( t \right) = \left( g \ast y \right) \left( t \right) $$
Minimizes:
$$ \mathbb{E} \left[ { \left( x \left( t \right) - \hat{x} \left( t \right) \right)}^{2} \right] $$
Since we defined the Winer Filter $ g \left( t \right) $ by the Convolution Operation we defined it as LTI.
It's not that the Wiener filter "ensures that the filtering is linear time invariant". The Wiener filter is linear and time-invariant. Linearity and time-invariance are just properties of filters.
For example, suppose you have a blue shirt. What you are asking about the Wiener filter is something similar to asking "how does the shirt ensure that its color is blue?". You see, it doesn't make much sense. It's blue just because you made it that way.
The Wiener filter is designed to be LTI, so that its output is the convolution between the input signal and its impulse response. LTI filters have many interesting and practical properties, so that's the reason why we often deal with these type of filters. Remember that this is why this is called 'linear estimation'. If you were to apply a more sophisticated non-linear estimation procedure, then the Wiener filter would have to be replaced by something else.
The Wiener filter is the solution to what's known as the optimal filtering problem: optimally estimate (with minimum mean squared error) a desired signal $y[n]$ from another signal $x[n]$ which is correlated to $y[n]$ but also corrupted with noise.
Now, if you assume that the desired signal $y[n]$ is a WSS (wide sense stationary) random process, and that $x[n]$ also is WSS, then you can assume that the optimal filter (Wiener filter) which estimates $y[n]$ from input $x[n]$ can be an LTI filter with a fixed impulse response of $w[n]$ (mostly taken to be an FIR system).
Then you can construct a mathematical formulation of the optimal filtering problem, as did @Royi in his anwser, to find out the particular solution; (i.e., the optimal Wiener filter $w[n]$) based on the given statistics of the input $x[n]$ and the desired signal $y[n]$ which is described as:
$$ w_o = R_{xx}^{-1} p_{xy} $$
Where $R_{xx} = \mathcal{E} \{ \bar{x}[n] \bar{x}[n]^{H} \}$ is the $N \times N$ autocorrelation matrix of the $N \times 1$ input signal vector $\bar{x}[n]= [x(n), x(n-1),...,x(n-N+1)]^{H}$ and the vector $p_{xy}$ is the cross-correlation between the input vector $\bar{x}[n]$ and single sample $y[n]$. These matrices are fixed in time when the signals are WSS, therefore the filter $w_o$ will be fixed in time hence LTI.
In this setting, the LTI assertion is implicitly made during the WSS assumtion of the random processes involved. Otherwise you may not end up with an LTI system which could solve the minimum mean square estimation problem.
Indeed, when the desired response $y[n]$ is not WSS, the Wiener filter becomes a tracking filter, implemented in an adaptive form (such as LMS) which has a time-varying impulse response; an LTI $w[n]$ cannot then be the solution of optimal filtering problem in that case.
