Least Squares Linear Phase FIR Filter Design

Question

In explication ''the geometric interpretation of least squares'' Typically, the number of frequency constraints is much greater than the number of design variables (filter coefficients). In these cases, we have an overdetermined system of equations (more equations than unknowns). Therefore, we cannot generally satisfy all the equations, and are left with minimizing some error criterion to find the ``optimal compromise'' solution.'

• What is meant by ''constraints''

• What is the relation between constraints and the number of design variables (filter coefficients).

In this case must be using a least-squares approximation.But, are there other cases when constraints less than or equal to the number of design variables?

Answer 1

I think that the term "constraints" is not a very fortunate choice in this context, but what is meant is the number of frequency points that are specified:

$$H(\omega_i)\stackrel{!}{=}D(\omega_i),\qquad i=1,2,\ldots,K\tag{1}$$

where $H(\omega_i)$ is the actual frequency response evaluated at frequency $\omega_i$, $D(\omega_i)$ is the desired response at $\omega_i$, and $\stackrel{!}{=}$ means that this is not an actual equality but we're trying to minimize some norm of the error $H(\omega_i)-D(\omega_i)$ over all frequencies $\omega_i$, $i=1,2,\ldots,K$.

Generally, the number of "equations" in $(1)$ is much greater than the number of filter coefficients $N$. In practice one often choses $K$ $5$ to $10$ times greater than $N$. This is why you get an overdetermined system of equations which can be solved in a least-squares (or any other) sense.

If $K<N$ you have an interpolation problem, which means that normally all equations can be satisfied with equality, but you have no control over what happens in between the specified frequency points $\omega_i$. Also, if $K<N$ the filter coefficients are not completely specified because you have an underdetermined system of equations. Such a system can be solved with an additional requirement such as that the vector of filter coefficients has minimum norm. But traditionally, filter design problems are not formulated as underdetermined systems of linear equations.