I am having little difficulty to understand that why most of the adaptive algorithms use error power or addition of error power as cost function/minimization criterion. I have read that minimization of error power criterion gives us best results from estimation mean point of view. I am not being able to grasp that either. So can someone please tell me in simple language what are the actual reasons to use above criterion? Why cant we simply use error as the minimization criterion?
1
-
1$\begingroup$ Using only the error values (i.e. deviations from measurement to the model) won't work, since individual deviation values can cancel each other out. Example: Imagine one period of a sine wave, sampled equidistantly and symmetrically at an odd number of points). If you compare this to the correct model function (said sine wave), your summed up error between the model and the data will be zero as it should. If you would multiply your model by $-1$ (or introduce an $180^\circ$ shift and compare it to the data, the model will completely fail to explain the data, but will have a error sum 0, too. $\endgroup$– M529Commented Mar 12, 2016 at 14:14
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
In a very simple form: Gaussian-like noises are frequent, and derivatives of squared functions are easy to minimize.
In very simple words: error, directly, cannot be used as a criterion. You can minimize a signed function to $-\infty$, which is not really useful here. As least, you need a cost function with a lower bound.
Power (if by that you mean the squared error) is an increasing function of the error, and positive. So if you minimize power, you can expect to minimize, somehow, a measure of the error (more precisely its absolute value).
The first problem is that the absolute value is not differentiable around zero, and can be difficult to optimize (one may look for a zero in the derivative). And when can you very easily find a zero in a derivative? When it is linear, because since Laplace, Gauss, Seidel such systems can be solved efficiently (matrix inversion). What are the functions that yield linear derivatives? Squares, parabolas...
Basically, for a while, squared functions were the only ones one could solve easily. The standard mean is just the minimum of the sum of differences squared. Interestingly, in linear cases, unrelated disturbances can add with a Gaussian distributions. And Gaussian distributions are well-related to squared errors.
But you are interested, recent advances in optimization allow to optimize more easily other types of functions, including absolute value loss criteria or penalties.
answered Mar 12, 2016 at 13:41