I'm trying to review MLE (maximum likelihood estimation).
What does it mean when one model fit the data better than does a comptitor model?
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
You need a criteria for saying whats better (i.e. a risk function). You say estimator A is better than estimator B with respect to a given risk function if the risk of estimator A is better than estimator B.
In the case of estimation, the risk is often the mean square error, i.e. if $\hat{X}$ is your estimator and $X$ is the thing you're trying to estimate, $R(\hat{X}) = E[\lVert X - \hat{X} \rVert_2^2]$ where $\lVert \cdot \rVert_2$ is the 2-norm. Then, you say estimator A is better than estimator B if it has lower mean square error.
When working in the mean square error framework for deterministic parameters, you have the notion of efficient estimators, i.e. ones which meet the Cramer-Rao lower bound, which are the best you can do in some sense (as described on the wiki article).
-
$\begingroup$ I think you said the Least square methods. Then what does different between MLE and LSM? $\endgroup$ – gmotree May 26 '15 at 7:58
-
$\begingroup$ A MMSE method tries to find $\hat{X}$ such that $E[||X-\hat{X}||_2^2]$ is minimized. The MLE finds $\arg \max_{\theta} p(x;\theta)$. The reason why I mentioned mean square error is that its a common risk function, and you can use it to compare estimators. The MLE also usually has some nice (at least asymptotic) properties with respect to mean square error. $\endgroup$ – Batman May 26 '15 at 11:36
