A Tutorial on MM Algorithms
Top Cited Papers
- 1 February 2004
- journal article
- research article
- Published by Informa UK Limited in The American Statistician
- Vol. 58 (1), 30-37
- https://doi.org/10.1198/0003130042836
Abstract
Most problems in frequentist statistics involve optimization of a function such as a likelihood or a sum of squares. EM algorithms are among the most effective algorithms for maximum likelihood estimation because they consistently drive the likelihood uphill by maximizing a simple surrogate function for the log-likelihood. Iterative optimization of a surrogate function as exemplified by an EM algorithm does not necessarily require missing data. Indeed, every EM algorithm is a special case of the more general class of MM optimization algorithms, which typically exploit convexity rather than missing data in majorizing or minorizing an objective function. In our opinion, MM algorithms deserve to be part of the standard toolkit of professional statisticians. This article explains the principle behind MM algorithms, suggests some methods for constructing them, and discusses some of their attractive features. We include numerous examples throughout the article to illustrate the concepts described. In addition to surveying previous work on MM algorithms, this article introduces some new material on constrained optimization and standard error estimation.Keywords
This publication has 25 references indexed in Scilit:
- MM algorithms for generalized Bradley-Terry modelsThe Annals of Statistics, 2004
- Genomewide motif identification using a dictionary modelProceedings of the IEEE, 2002
- Quantile Regression via an MM AlgorithmJournal of Computational and Graphical Statistics, 2000
- [Optimization Transfer Using Surrogate Objective Functions]: RejoinderJournal of Computational and Graphical Statistics, 2000
- Direct Calculation of the Information Matrix via the EM AlgorithmJournal of the Royal Statistical Society Series B: Statistical Methodology, 1999
- Globally convergent algorithms for maximum a posteriori transmission tomographyIEEE Transactions on Image Processing, 1995
- Using EM to Obtain Asymptotic Variance-Covariance Matrices: The SEM AlgorithmJournal of the American Statistical Association, 1991
- On the Convergence Properties of the EM AlgorithmThe Annals of Statistics, 1983
- Modelling association football scoresStatistica Neerlandica, 1982
- An Iterative Technique for Absolute Deviations Curve FittingJournal of the American Statistical Association, 1973