I earnestly request apology if this question is inappropriate for the forum. The question has two parts one technical and the other is not technical. I would appreciate any response.

Let me consider a specific use of MDL: model order estimation. For radar/sonar it is equivalent to the estimation of the number of targets (location of target is not considered here). MDL has two parts: maximum likelihood (ML) estimate of the parameter and a penalty function. Many variations exist in the literature related to the penalty function but ML estimate is hardly touched as ML is considered optimal. The model, the ML estimate and the MDL estimate all fine with large sample size; but for small sample size or high noise it is not the case as the 'model' is not accurate. The model is valid only as an expected value which is an asymptotic value.

I have an algorithm which improves the estimate of the number of targets considerably at low number of sample size or higher noise. The algorithm uses concepts of quasi-maximum likelihood.

My questions are: 1) Can we prove that a asymptotically good model, whose parameter we want estimate, for small sample size can have better estimator than ML?

The non-technical question is:

2) Anyone interested in this problem or use MDL for number of targets?

Thanks a lot.

  • $\begingroup$ If an efficient estimator exists then ML will find it and nothing can exceed its performance. Also, ML estimates are unbiased for linear models. What model do you use? If it is linear model, then ML and least squares coincides and there are approaches to exceed the performance of them by using unbiased estimators. $\endgroup$ – Oliver Apr 13 '15 at 5:09

For lower number of samples the preferred algorithm is the Akaike Information Criteria (AIC). Both are not optimal in any way, though MDL is a consistent estimator of model order. The difference between them is the penalty term and not another estimator (other than ML)

I refer you to Petre Stoica paper on model order selection which gives a great summary of algorithms.

I am also interested to hear more about your algorithm.


1) For instance, A biased estimator may have better performance (such as variance) than ML. Don't know for your specific problem.

2) I suppose you are studying state estimation, and more speciaficaly, multi-target tracking problems as they arise in sonar and radar. In that case, finding the number of target is an essential problem and lots of people are working on it. Try to read some paper on PHD filters, whose main goal is to find an estimate on the mean number of target withing a given area.

www.ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4086095 is a gread practical introduction to PHD filter (in their most simple implementation)

Vo brothers are doing great work on this subject. As well as herriot-watt unviersity (Daniel Clark if my memory is good)

  • $\begingroup$ Thank you for your interest. I think when noise is associated we do not know the instances of the noise so the model is defined in terms of expectation. My question is related to small sample size so the validity of the model itself is interesting. The validity of the maximum likelihood is far way from being correct as the independence of the model parameters is in doubt in case of small sample size. My question refers to the solution of this problem. @Antoine the mean square error performance is also better not only variance. $\endgroup$ – Creator Apr 13 '15 at 20:02
  • $\begingroup$ I can't realy help you, but there is such thing that Cramer rao lower bound for non-independant population, maybe this could help you ? $\endgroup$ – Antoine Bassoul Apr 14 '15 at 8:23
  • $\begingroup$ by the way, I'm very interested in your poblem and the algorithm you have, I would appreciate any precision you could give $\endgroup$ – Antoine Bassoul Apr 14 '15 at 8:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.