Optimal Estimation of Parameters

Jorma Rissanen was a member of research staff in IBM Almaden Research Center from 1965 to 2001 and is currently Professor Emeritus at Technical University of Tampere, Finland. Among his main achievements are the introduction of the MDL principle for statistics, the invention of arithmetic coding and the introduction of variable-length Markov chains with the associated Algorithm Context. He has received many awards, including the 2007 Kolmogorov medal from the CLRC, University of London, and the 2009 Shannon Award from the Information Theory Society. He received two Outstanding Innovation Awards from IBM in 1980 and 1988 and a Corporate Award in 1991.

• 1. Introduction; 2. Coding; 3. Basics of information; 4. Modeling problem; 5. Other optimality properties; 6. Interval estimation; 7. Hypothesis testing; 8. Denoising; 9. Sequential models; Appendix A. Elements of algorithmic information; Appendix B. Universal prior for integers.

'The minimum description length (MDL) principle is a very universal principle of statistical modeling in estimation, prediction, testing, and coding. Jorma Rissanen, the pioneer of the MDL principle, evolves a new theory to reach the most general and complete notion, which he calls the complete MDL principle. In this book the author derives it by introducing the key notion of maximum capacity. The most fundamental methods of estimation such as maximum likelihood estimation and the MDL estimation are naturally derived as the maximum capacity estimators, and their optimality is justified within a unifying theoretical framework. Through the book, readers can revisit the meaning of estimation from the author's very original viewpoint, and will enjoy the most advanced version of the MDL principle.' Kenji Yamanishi, University of Tokyo