4
$\begingroup$

In many research articles the performance of an estimation method is compared to that of the ML estimation performance. If the performance of the method does not achieve the ML estimation performance, then the method is 'suboptimal' or not good enough. I don't quite remember what is the meaning of ML estimation performance and sub-optimal or optimal. When is a method optimal and optimal with respect to what? I am following the book titled: "Statistical signal processing Vol 1 by S. Kay". As a beginner and self-learner it is quite hard to grasp the actual implication of the terms -- achieving ML estimation performance, optimal and sub-optimal.

Can somebody please explain an intuitive meaning to these terms? Thank you.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

Coming up with a "good" estimator for a parameter of interest is not an easy task because it is important to define what good means. There are many ways of defining it, depending on your application. "Good" and "optimal" mean the same thing.

Maximum likelihood (ML) estimation arises from one specific intuitively pleasing way of defining good: an estimator is good if it maximizes the probability of observing the given data. Moreover, maximum likelihood (ML) estimators have some desirable properties such as asymptotic unbiasedness and functional invariance, and asymptotic normality which makes it convenient to analyze their performance theoretically. This is why ML estimators are used as a benchmark by many authors.

An estimator achieves ML performance and is optimal if it provides the same nice theoretical properties of an ML estimator. An estimator is sub-optimal if it provides worse theoretical performance than an ML estimator.

Interesting additional reading: The Epic Story of Maximum Likelihood

Improve this answer
$\endgroup$
3
  • $\begingroup$ (1) In general - it is hard to know. In some cases you may be able to use simple calculus (set first derivative to zero, and check sign of second derivative) to locate the maximizer. (2) Unbiased means the expected value of the estimator is equal to the true value. (3) No. ML estimation procedure can be used with any arbitrary probability model. $\endgroup$
    – Atul Ingle
    Oct 9, 2017 at 2:42
  • $\begingroup$ @AtulIngle Thanks for the link; very interesting read. $\endgroup$
    – MBaz
    Oct 9, 2017 at 14:31
  • $\begingroup$ If I trust your definition of blind v/s non-blind, I would think MLE is non-blind---you are given a bunch of data and from that you estimate an unknown parameter of interest. $\endgroup$
    – Atul Ingle
    Oct 10, 2017 at 22:13
1
$\begingroup$

There are 3 major reasons in my opinion which makes the Maximum Likelihood Estimator so popular:

  1. Intuition
    It is very clear what's the logic behind this method and what you maximize. It makes sense even if you have little knowledge in probability and statistics.
  2. Properties
    It has great properties (Some are only asymptotically, namely with many samples) such as consistency and efficiency. This means it always makes some sense to use it.
  3. Easy to Derive
    It is usually pretty easy to derive the optimization problem defined by the MLE and in some cases it is also simple to solve it.

Now, many parameter estimation methods are compared to the properties of the MLE - Efficiency and Consistency.
If they exhibit similar performance than they are as optimal as the MLE.
Otherwise, they are not, as expected.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.