1
$\begingroup$

Can somebody please provide an intuitive answer or reference for the following questions?

Q1: Dependence of estimation performance on number of data points -- I could not find any information whether the estimation performance of Adaptive filters such as Least Mean Square (LMS), Constant Modulus Algorithm (CMA) and Kalman Filters depend on the number of data points or not. Is there any information whether the mean square error between the actual and estimated parameters reduces with the increase in the number of data points or not?

Q2: Dependence of convergence on number of data points -- For instance, information such as if convergence of these adaptive filters (or in general) depends on the number of data points i.e., if these require a large number of data points to have good estimation performance.

|improve this question
$\endgroup$
  • $\begingroup$ Number of data points means signal length, or do you mean filter length (number of tap weights) ? $\endgroup$ – Fat32 Aug 21 '17 at 19:10
  • $\begingroup$ I mean signal length $\endgroup$ – Srishti M Aug 21 '17 at 19:12
2
$\begingroup$

Q1: Dependence of estimation performance on number of data points

Since LMS and RLS are adaptive filters, their estimation performance improves as the number of their iterations increase. Hence more data points will make their outputs closer to the expected performance, until the convergence is achieved (this requires either a WSS data or a slowly changing, statistically, nonstationary data, so that filter can track it's statistical character). Once the filter is operating in convergence conditions, then there won't be any improvements in its output, providing what is called as the (minimum) steady-state error. This steady-state error depends on a number of factors but not on the data length.

But until the convergence is achieved, the estimation error decreases as more data points processed. However most typical adaptive filter users would be interested in its steady state (convergent) results rather than the transient response.

Note that increased data points can either come from a longer observation or from higher sampling rate, the results of which can be different. For example for a Kalman filter (in its extended mode) with mechanical applications, an increase in sampling rate can actually improve the steady state error as well.

Q2: Dependence of convergence on number of data points

For both LMS, RLS and Kalman filters, convergence primarily depends on the number of data points being processed; i.e., number of iterations. However this convergence rate is different for different filter types, reflecting their complexity and/or sophistication. The simple LMS filter has the slowest rate of convergence (roughly 10 to 20 times the filter tap weight length) whereas the RLS and Kalman filters display a convergence rate of roughly $2$ times the filter length (Haykin_Adaptive Filter Theory) for WSS inputs.

|improve this answer
$\endgroup$
  • $\begingroup$ Your welcome! Coming to iterative MLE, honestly I haven't use it so I cannot state an opinion on it, but the above answers actually applies to many (if not all) iterative algorithms. However the adaptive filters have the distinctive feature that they have feedback. They feed their current iteration errors back into the next output computation. It's this architecture that makes them adaptive. Any algorithm that has a similar implementation architecture will also be adaptive, hence the answer will apply to them as well, unless otherwise stated by the particular algorithm. $\endgroup$ – Fat32 Aug 21 '17 at 20:58
  • 2
    $\begingroup$ tough question :-) Recently, here, there was a debate on calling Kalman filters as adaptive or not. @StanleyPawlukiewicz indicated that Linear Kalman filters are not adaptive because all gain computations can be done offline and independent of the incoming data. I agree on this. But extended Kalman filters update their matrices based on their states, which depend on input data so they are adaptive. Note that making updates is necessary but not sufficient for adaptivity. those updates must be based on incoming data, either in feedback or in feedforward form. LMS,RLS use feedback $\endgroup$ – Fat32 Aug 21 '17 at 21:15
  • 1
    $\begingroup$ Note that Kalman filters in signal processing are based on modifications and interpretations performed by Godard and Kailath, as described in Haykin's book. However historically Kalman filters were considered from a mechanical applications point of view, which makes comparisons difficult between those two different interpretations. $\endgroup$ – Fat32 Aug 21 '17 at 21:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.