Variational Bayesian Learning Theory

Shinichi Nakajima is a senior researcher at Technische Universität Berlin. His research interests include the theory and applications of machine learning, and he has published papers at numerous conferences and in journals such as the Journal of Machine Learning Research, the Machine Learning Journal, Neural Computation, and IEEE Transactions on Signal Processing. He currently serves as an area chair for NIPS and an action Editor for Digital Signal Processing. Kazuho Watanabe is a lecturer at Toyohashi University of Technology. His research interests include statistical machine learning and information theory, and he has published papers at numerous conferences and in journals such as the Journal of Machine Learning Research, the Machine Learning Journal, IEEE Transactions on Information Theory, and IEEE Transactions on Neural Networks and Learning Systems. Masashi Sugiyama is Director of the RIKEN Center for Advanced Intelligence Project and Professor of Complexity Science and Engineering at the University of Tokyo. His research interests include the theory, algorithms, and applications of machine learning. He has written several books on machine learning, including Density Ratio Estimation in Machine Learning (Cambridge, 2012). He served as program co-chair and general co-chair of the NIPS conference in 2015 and 2016, respectively, and received the Japan Academy Medal in 2017.

• 1. Bayesian learning; 2. Variational Bayesian learning; 3. VB algorithm for multi-linear models; 4. VB Algorithm for latent variable models; 5. VB algorithm under No Conjugacy; 6. Global VB solution of fully observed matrix factorization; 7. Model-induced regularization and sparsity inducing mechanism; 8. Performance analysis of VB matrix factorization; 9. Global solver for matrix factorization; 10. Global solver for low-rank subspace clustering; 11. Efficient solver for sparse additive matrix factorization; 12. MAP and partially Bayesian learning; 13. Asymptotic Bayesian learning theory; 14. Asymptotic VB theory of reduced rank regression; 15. Asymptotic VB theory of mixture models; 16. Asymptotic VB theory of other latent variable models; 17. Unified theory.

'This book presents a very thorough and useful explanation of classical (pre deep learning) mean field variational Bayes. It covers basic algorithms, detailed derivations for various models (eg matrix factorization, GLMs, GMMs, HMMs), and advanced theory, including results on sparsity of the VB estimator, and asymptotic  properties (generalization bounds).' Kevin Murphy, Research scientist, Google Brain