Signal Processing Stack Exchange is a question and answer site for practitioners of the art and science of signal, image and video processing. It only takes a minute to sign up.
Sign up to join this community
I would like to limit discussion to discrete time causal Wiener filter.
By definition it is an FIR filter, and it also has optimality under Mean Square Error criterion within the class of linear filters.
How do we know that there isn't a better IIR filter (something with a feedback loop), which is also linear?
You're right, there usually is a better IIR filter (if you have enough data). The discrete-time Wiener filter is not "by definition" FIR. It is common to constrain the filter to the FIR case because it's often more straightforward to implement, and because such a filter can be made adaptive more easily. Also, in practice you often want to consider only a finite data window, which makes the optimum Wiener filter FIR.
However, you don't need to impose the constraint that the filter length be finite. The most general causal discrete-time Wiener filter for data available over the infinite past is an IIR filter.
A derivation of the FIR and the IIR cases can be found (among others) in Digital Signal Processing by Proakis and Manolakis.
EDIT: If we only have a finite amount of data available, the optimum Wiener filter has a finite impulse response. Assume we have $N$ data points (one current and $N-1$ consecutive past data points), then the best we can do with a linear time-invariant filter is to linearly combine those $N$ data points, by multiplying each data point by a coefficient and adding them up:
$$y[n]=\sum_{k=0}^{N-1}h[k]x[n-k]\tag{1}$$
The coefficients $h[k]$ are optimized such that the mean square error between the filter output $y[n]$ and some desired signal is minimized. The $N$ coefficients $h[k]$ are the impulse response of the corresponding FIR Wiener filter.
