The quadratic performance surface of an adaptive filter is a paraboloid. Its minimum can be found wherever the gradient is zero. However, since there are two types of paraboloids (elliptical and hyperbolic), is there a way to tell if the minimum detected is a global minimum or just a saddle point?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
The quadratic surface is determined by the autocorrelation matrix of the data, which is always positive definite or positive semi-definite. This means that any stationary point is always a minimum. In the worst case, this minimum is not unique if the matrix is singular, but it can never be a saddle point.
-
$\begingroup$ Can you please explain how it can never be a saddle point? I assumed that if the quadratic surface is a hyperbolic paraboloid (in which one parabola is turning upwards and the other downwards), a saddle point exists where the gradient is zero. $\endgroup$ May 3, 2015 at 19:54
-
1$\begingroup$ @user1832413: If you have a quadratic form $x^TAx+b^Tx$, and if the matrix $A$ is positive (semi-)definite (as is the case with an autocorrelation matrix), then the function defined by the quadratic form is convex and has a minimum. A saddle point can only occur if the matrix $A$ is indefinite. $\endgroup$– Matt L.May 3, 2015 at 20:00
-
$\begingroup$ for the LMS adaptive filter (Widrow), it can't be a saddle point because the function is magnitude square error which can't go negative. a minimum point close to zero can only go up on the sides, not down. $\endgroup$ May 4, 2015 at 1:54