- Here I expected $y(n)$ is to be computed by convolving $x(n)$ with $h(n)$, but in the equation given by Wikipedia it is shown as a matrix multiplication
$y(n) = h^H(n).x(n)$. Are these two operations(convolution and matrix multiplication) same here?.
The system is an FIR system, so the vector multiplication here is equivalent to convolution --- for one time instant output. Note that $x(n)$ is a vector o the current and last $N$ samples of $x(n)$.
- Why do we need a Hermitian transpose rather than normal transpose in the above equation?
Because the signal is assumed to be complex.
- Cost function is defined as $E\{|e(n)|^2\}$. Why do we need expectation operator here? Why not normal Mean Square Error(MSE)?
Because the error is assumed to be stochastic. We don't want to minimize for one realization, we want to minimize for all realizations.
- Does LMS filter converge to all type of input signal?
You don't believe the proof? :-)
Provided the assumptions made by the proof are met, then the LMS algorithm converges for all signals.
The convergence is only convergence in the mean: what can happen is the filter gets close to the optimal value, but bounces around a bit. It's not guaranteed to stay precisely on the optimal value.
The main constraint is on the value of the step size, $\mu$. It needs to be big enough to ensure convergence and small enough to not diverge.
The picture below shows a couple of examples. One where the convergence is slow but steady (low, but high enough $\mu$) and the other where $\mu$ is a little higher where the convergence bounces around a little.
- I understand LMS make use of Stochastic Gradient Descent and the update equation for the same is $\hat{h}(n+1)=\hat{h}(n)+\mu x(n)e^*(n)$, what would be the update equation if we would have used a normal Gradient Descent?
The main difference is that stochastic gradient descent uses sample-by-sample updates, rather than a larger data set. See this discussion on Quora for more details.
