4
$\begingroup$

I understand that mathematical motivation behind Bayesian Information Criterion (BIC), Minimum description length (MDL), Akaike information criterion (AIC) are different but I am assuming that there are several applications where all three are equally applicable. I am interested to know the applications of these algorithm in as many areas as possible. I can only guess that it is used in estimation of number of targets in radar. But I can also imagine that such a general concepts can be applied in many other problems.

May I request to users of these algorithms to write a few lines on how they are using and are they using it in some products?

Any comments would be appreciated.

$\endgroup$

2 Answers 2

1
$\begingroup$

The Bayesian Information Criterion (BIC), Akaike Information Criterion (AIC), and Minimum Description Length (MDL) are all model selection criteria used to choose among multiple candidate models. They balance the fit of the model to the data against the complexity of the model. Here's a quick overview of their mathematical motivations:

  • AIC: Developed by Hirotugu Akaike, it aims to find the model that best explains the data with a minimal number of parameters. It penalizes overfitting but tends to favor slightly more complex models compared to BIC.

  • BIC: Developed from Bayesian principles, BIC also penalizes model complexity but does so more harshly than AIC. It assumes a prior distribution over the model's parameters.

  • MDL: Minimum Description Length principle is rooted in information theory and focuses on finding the model that most succinctly represents the data. Essentially, it's about compressing the data as much as possible.

Their applications span various fields. Here are some of them:

  1. Statistical Modeling and Regression Analysis: These criteria help in choosing between models with different numbers of parameters.

  2. Time Series Analysis: Used in selecting the order of ARIMA models or when determining the number of lags in a regression model.

  3. Machine Learning: For instance, in deciding the number of clusters in K-means clustering or in pruning decision trees.

  4. Neuroscience: For example, in choosing models of neuron firing rates or in source localization in EEG.

  5. Psychology: In psychometrics, for deciding between different cognitive models.

  6. Economics: For choosing among alternative econometric models.

  7. Phylogenetics: In determining evolutionary trees from molecular data.

  8. Medical Research: For instance, in modeling the progression of diseases or in modeling response to treatment.

  9. Astronomy: As you correctly pointed out, model selection criteria can be used in problems like estimating the number of sources or targets, such as in radar or in determining the number of celestial objects contributing to a signal.

  10. Ecology: In estimating species richness or in modeling population dynamics.

  11. Bioinformatics: For instance, in selecting models for protein structure or in gene expression analysis.

Users across various disciplines often employ these criteria in bespoke software tools or as part of broader statistical packages like R, Python's Scikit-learn, or MATLAB. If you're looking for personal testimonies, academic journals, forums, or discussion boards dedicated to specific research areas might be a good place to look.

$\endgroup$
0
$\begingroup$

One classic application is setting the k parameter in K-Means Clustering and Gaussian Mixture Model (GMM) Clustering.

You may use them in Maximum Likelihood estimation as well.
For instance, in Linear Regression models you may count the number of parameters and the variance of the residuals (See Wikipedia AIC - Counting Parameters) and use the AIC / MDL / BIC to compare different models with the different parameters number.

$\endgroup$
2
  • $\begingroup$ What about non clustering applications? $\endgroup$ Commented Jul 8, 2023 at 5:56
  • $\begingroup$ @GeorgeIrwin, I added ML based example. $\endgroup$
    – Royi
    Commented Jul 8, 2023 at 7:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.