5

The Jacobian is not computed numerically but analytically and then just evaluated. The frequency response of the IIR filter is $$H(e^{j\omega})=\frac{b_0+b_1e^{-j\omega}+\ldots+b_Me^{-jM\omega}}{1+a_1e^{-j\omega}+\ldots+a_Ne^{-jN\omega}}=\frac{B(e^{j\omega})}{A(e^{j\omega})}\tag{1}$$ Now you need the derivative with respect to the filter coefficients: $$\...


3

Let's solve a more general problem (Least Squares with Linear Equality Constraints): $$ \begin{alignat*}{3} \arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{2} \\ \text{subject to} & \quad & C x = d \end{alignat*} $$ The Lagrangian is given by: $$ L \left( x, \nu \right) = \frac{1}{2} \left\| A x - b \right\|_{2}^{2} + {\...


3

I suggest this reference regarding the comparison between least-squares and Kalman filters : Fundamentals of Kalman Filtering: A Practical Approach by P. Zarchan & H. Mussof Especially Chapter 3 (Recursive Least-Squares Filtering) and Chapter 4 (Polynomial Kalman Filters). In Chapter 4, the authors show that the discrete (time) n-th order polynomial ...


1

Ideas: 1) Find the minimum eigen vector of $R_1$ and assign this to $w$. This will minimize $ J_1$ , doesn't maximize $J_2$. But when $R_1$ and $R_2$ are positive definite or full rank matrices and computation is an issue, this is a decent solution. 2) Form a new objective $J_1 - J_2$ and minimize this with the given constraint 3) Try and formulate as ...


1

I will give you a hint: you can first relax this problem to be a convex optimization problem by editing the second constraint as $$Xw <= \vec{1}$$ where the inequality is elementwise, then form the dual problem or the lagrangian as it is known popularly $$ w^TRw + \lambda^T(Xw -1) \tag{1}$$ where $$\lambda <=\vec{0}$$ Differencate (1) with respect ...


1

$$ J(h) = \int_{R^+UR-}|e^Th-F_d|^2d\omega\tag{1}\\ = \int(e^Th-F_d)^H(e^Th-F_d)d\omega\\ = \int((e^Th)^H(e^Th) + F_d^HF_d -(e^Th)^HF_d - F_d^He^Th)d\omega\\ = h^H(\int(e^T)^He^Td\omega) h + \int |F_d|^2d\omega -\int (2 Re\{(e^Th)^HF_d\})d\omega $$ The second term in above integral is the integral of $L_2$ norm of $F_d$ in the region $R^+ U R^-$. Since $F_d(...


1

The problem that recursive least squares (RLS) can solve can be formulated as recursively solving for $\hat{\theta}$, such that it is the least squares solution to $$ \hat{\theta}_n = \arg\min_x \sum_{k=0}^n w[k]\,\|z[k] - \phi[k]^\top x\|_2^2, $$ where $w[k]$ are weights, $z[k]$ and $\phi[k]$ are known and $z[k]$ is assumed to be generated by using $\phi[...


1

The reason x[n] must be white is because the solution will effectively spectrally weight the channel response based on the amount of energy present in each spectral frequency location. A white noise source provides equal weight to all frequencies. If energy is not present in any particular frequency bin, a proper solution cannot be found for that frequency. ...


1

$$(A^TA)^{-1}A^T(y + noise) = \hat x$$ where $noise$ is vector of the same size as $y$ with all elements equal to unknown constant. That is all. You can get your estimated solution as a function of noise mean position in explicit form.


1

I can add that LMS algorithm has a sample-based update.


Only top voted, non community-wiki answers of a minimum length are eligible