In my understanding, I let make a example.

For example, you can see in the following picture.

Consequently, We want to find ^sigma. and We have already known the observation data(which is random data such as C) and bin(0.3).

To know "the estimated standard deviation ^sigma", First of all,

  1. Find Likelihood for each bins with observed data( for example, data of C).
  2. Find Maximum Likelihood among previous Likelihood values(from 1).
  3. the Maximum Likelihood is "the estimated standard deviation ^sigma". Am I wrong? In other words, the Maximum Likelihood (^sigma) represent all bins's Likelihood values.

enter image description here


Normally you start with a probabilistic model of your data (observations). That model includes a noise term which has some variance or standard deviation which can vary over time or can be fixed. Then you write up the joint pdf of the data, normally written something like this $p(\boldsymbol{x}|\theta)$, where $\theta$ is a vector that includes all the parameters in the model including the noise variance. Then in likelihood estimation the pdf is viewed as a function of $\theta$ (data is fixed). The parameter vector that maximizes this function is the most likely parameter vector in the sense that it maximizes the probability that the model has generated the data. What your plots seem to indicate is that the underlying estimator is biased which means that the expected estimate is not the true value.

The ML estimator is a function. It takes as input your data $\boldsymbol{x}$ and provides a value (the estimate) $\hat{\theta}$. Your data is random, it is governed by a pdf. Because you plug random data into your estimator your estimate will also be a random variable governed by some pdf. I think this is what is illustrated in those images. Those plots illustrate the pdf of the estimate. It is then of course interesting to know if the expected value (the mean) of the estimate then is equal to the true value.

  • 1
    $\begingroup$ I'm not familiar with image processing. Maybe you can read your references and then ask questions here whenever you read something that you don't understand. Step-by-step. $\endgroup$ – niaren May 20 '15 at 6:23
  • 1
    $\begingroup$ Are you asking about the difference between ML estimation and MAP estimation? By definition an ML estimator $\hat{\theta}$ maximizes $p(\boldsymbol{x}|\theta)$ or $\hat{\theta} = \underset{\theta}{\operatorname{argmax}} p(\boldsymbol{x}|\theta)$ $\endgroup$ – niaren May 20 '15 at 8:37
  • 1
    $\begingroup$ Do you have different models? or are you taking about parameter estimation in a model? Please clarify $\endgroup$ – niaren May 20 '15 at 8:46
  • 1
    $\begingroup$ I have added a few lines very briefly explaining the plots as I understand them. $\endgroup$ – niaren May 20 '15 at 9:15
  • 1
    $\begingroup$ Sorry, what I mean by writing "it is governed by pdf" is that the random variable is associated with a pdf. The output of the estimator is an estimate of the standard deviation in your example. $\endgroup$ – niaren May 21 '15 at 6:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.