Cost function for LTI system identification

Question

I am currently reading and trying to understand a paper (Kulkarni and Colburn, 2004) that utilizes system identification methods to approximate head-related transfer functions.

The general approach is to

1. Compute an autoregressive (all-pole-) estimate of the transfer function using the autocorrelation method for linear prediction.
2. Use the AR estimate as a starting point to compute a pole-zero-representation of the system transfer function iteratively.
3. Evaluate the result of the estimation process on a logarithmic scale (Error in dB).

For the iterative procedure, the authors are proposing a cost function

$\hat{C} = \frac{1}{2\pi}\int_{-\pi}^{\pi}|H(e^{j\omega})A(e^{j\omega}) - B(e^{j\omega})|^2 d\omega $,

where $\hat{H}$ is the system transfer function, $A$ is the DTFT of the recursive coefficients and $B$ the DTFT of the transversal coefficients. I understand this approach originates from this paper (Kalman, 1958).

This cost function is then extended for the iterative process as

$\hat{C}_i = \frac{1}{2\pi}\int_{-\pi}^{\pi}|\frac{H(e^{j\omega})A_i(e^{j\omega})}{A_{i-1}(e^{j\omega})} - \frac{B_i(e^{j\omega})}{A_{i-1}(e^{j\omega})}|^2 d\omega $,

where the index $i$ denotes variables corresponding to the $i$th iteration. The iterative modification originates from this paper (Steiglitz and McBride, 1965).

In order to find a solution on a decibel scale, a weighting function $W$ is introduced:

$\hat{C}_i = \frac{1}{2\pi}\int_{-\pi}^{\pi} |W_i(e^{j\omega})|^2 |\frac{H(e^{j\omega})A_i(e^{j\omega})}{A_{i-1}(e^{j\omega})} - \frac{B_i(e^{j\omega})}{A_{i-1}(e^{j\omega})}|^2 d\omega $,

which is introduced in the paper as

$W_i(e^{j\omega}) = \frac{\log\left( H(e^{j\omega})\right) - \log\left( \frac{B_{i-1}(e^{j\omega})}{A_{i-1}(e^{j\omega})}\right)}{|H(e^{j\omega})A_{i-1}(e^{j\omega}) - B_{i-1}(e^{j\omega})|^2}$.

I understand this weighting function is the squared error in logarithmic scale between true and approximated transfer function, divided by the first cost function.

However, i have trouble understanding the process of arriving at the iterative cost function for several reasons. I would like to ask the following questions:

• Why is the first cost function $\hat{C}$ preferred to, say, $\frac{1}{2\pi}\int_{-\pi}^{\pi} |H(e^{j\omega}) - B(e^{j\omega})/A(e^{j\omega})|^2$ in the first place? what does it do?
• In the iterative cost function $\hat{C}_i$, what is the purpose of dividing by the recursive proportion of the last transfer function estimate?
• For what reason is a weighting function introduced for logarithmic error minimization, rather than just using its numerator as a cost function directly?

I would really appreciate any help or pointers into the right direction.

Answer 1

The chosen cost function is the mean squared error, i.e., the integral over a squared magnitude of the difference between frequency responses. The function

$$E(e^{j\omega})=H(e^{j\omega})-\frac{B(e^{j\omega})}{A(e^{j \omega})}\tag{1}$$

depends on frequency, so you can't minimize it directly, unless you want to minimize it for exactly one frequency $\omega$, which is of course pointless. You can choose several error measures depending on $E(e^{j\omega})$ given in $(1)$. Two common choices are

$$\varepsilon_1=\max_{\omega}W(\omega)|E(e^{j\omega})|\tag{2}$$

and

$$\varepsilon_2=\int_{0}^{\pi}W(\omega)|E(e^{j\omega})|^2d\omega\tag{3}$$

with some positive weighting function $W(\omega)$. Note that $\varepsilon_1$ and $\varepsilon_2$ given by $(2)$ and $(3)$ do not depend on frequency.

The authors of the paper you refer to chose to minimize the weighted mean square error given by $(3)$. However, instead of using the linear difference $(1)$, they chose to minimize the average logarithmic difference, i.e., the average error on a dB scale.

The problem with the minimization of $(3)$ is that for IIR filters, it results in a non-linear optimization problem, which is much harder to solve directly, and which might also have locally optimal solutions that are far from the global optimum. The cost function $\hat{C}$ in your question is linear in the filter coefficients. The point of the iteration is now to solve a sequence of linear minimization problems (which is simple, just solve a system of linear equations) in order to compute the solution of the originally non-linear optimization problem (the minimization of $(3)$). Note that if convergence is achieved, then $A_{i-1}(e^{j\omega})=A_{i}(e^{j\omega})$, so the cost function $\hat{C}_i$ is identical to the cost function of the original non-linear problem. Yet, only linear minimization problems are solved in each iteration.

I'm not sure I completely understand your last question, but the weighting function is there to change to problem from minimizing the average squared difference between frequency responses to the average squared differences between the logarithms of the magnitude responses. The necessary weighting function is unknown, but if the procedure converges - note that there is no guarantee that it does in all cases - the final weight function is such that the mean squared logarithmic error is minimized. Note that this latter problem is highly non-linear, whereas the proposed procedure only solves linear subproblems.