# Tag Info

## Hot answers tagged least-squares

5

The Jacobian is not computed numerically but analytically and then just evaluated. The frequency response of the IIR filter is $$H(e^{j\omega})=\frac{b_0+b_1e^{-j\omega}+\ldots+b_Me^{-jM\omega}}{1+a_1e^{-j\omega}+\ldots+a_Ne^{-jN\omega}}=\frac{B(e^{j\omega})}{A(e^{j\omega})}\tag{1}$$ Now you need the derivative with respect to the filter coefficients: $$\... 3 Let's solve a more general problem (Least Squares with Linear Equality Constraints):$$ \begin{alignat*}{3} \arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{2} \\ \text{subject to} & \quad & C x = d \end{alignat*} $$The Lagrangian is given by:$$ L \left( x, \nu \right) = \frac{1}{2} \left\| A x - b \right\|_{2}^{2} + {\...

3

I suggest this reference regarding the comparison between least-squares and Kalman filters : Fundamentals of Kalman Filtering: A Practical Approach by P. Zarchan & H. Mussof Especially Chapter 3 (Recursive Least-Squares Filtering) and Chapter 4 (Polynomial Kalman Filters). In Chapter 4, the authors show that the discrete (time) n-th order polynomial ...

1

Ideas: 1) Find the minimum eigen vector of $R_1$ and assign this to $w$. This will minimize $J_1$ , doesn't maximize $J_2$. But when $R_1$ and $R_2$ are positive definite or full rank matrices and computation is an issue, this is a decent solution. 2) Form a new objective $J_1 - J_2$ and minimize this with the given constraint 3) Try and formulate as ...

1

I will give you a hint: you can first relax this problem to be a convex optimization problem by editing the second constraint as $$Xw <= \vec{1}$$ where the inequality is elementwise, then form the dual problem or the lagrangian as it is known popularly $$w^TRw + \lambda^T(Xw -1) \tag{1}$$ where $$\lambda <=\vec{0}$$ Differencate (1) with respect ...

1

$$J(h) = \int_{R^+UR-}|e^Th-F_d|^2d\omega\tag{1}\\ = \int(e^Th-F_d)^H(e^Th-F_d)d\omega\\ = \int((e^Th)^H(e^Th) + F_d^HF_d -(e^Th)^HF_d - F_d^He^Th)d\omega\\ = h^H(\int(e^T)^He^Td\omega) h + \int |F_d|^2d\omega -\int (2 Re\{(e^Th)^HF_d\})d\omega$$ The second term in above integral is the integral of $L_2$ norm of $F_d$ in the region $R^+ U R^-$. Since $F_d(... 1 The problem that recursive least squares (RLS) can solve can be formulated as recursively solving for$\hat{\theta}$, such that it is the least squares solution to $$\hat{\theta}_n = \arg\min_x \sum_{k=0}^n w[k]\,\|z[k] - \phi[k]^\top x\|_2^2,$$ where$w[k]$are weights,$z[k]$and$\phi[k]$are known and$z[k]$is assumed to be generated by using$\phi[...

1

The reason x[n] must be white is because the solution will effectively spectrally weight the channel response based on the amount of energy present in each spectral frequency location. A white noise source provides equal weight to all frequencies. If energy is not present in any particular frequency bin, a proper solution cannot be found for that frequency. ...

1

$$(A^TA)^{-1}A^T(y + noise) = \hat x$$ where $noise$ is vector of the same size as $y$ with all elements equal to unknown constant. That is all. You can get your estimated solution as a function of noise mean position in explicit form.

1

I can add that LMS algorithm has a sample-based update.

Only top voted, non community-wiki answers of a minimum length are eligible