PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A Gradient-Based Algorithm Competitive with Variational Bayesian EM for Mixture of Gaussians
Mikael Kuusela, Tapani Raiko, Antti Honkela and Juha Karhunen
In: International Joint Conference on Neural Networks, 14-19 June 2009, Atlanta, USA.


While variational Bayesian (VB) inference is typically done with the so called VB EM algorithm, there are models where it cannot be applied because either the E-step or the M-step cannot be solved analytically. In 2007, Honkela et al. introduced a recipe for a gradient-based algorithm for VB inference that does not have such a restriction. In this paper, we derive the algorithm in the case of the mixture of Gaussians model. For the first time, the algorithm is experimentally compared to VB EM and its variant with both artificial and real data. We conclude that the algorithms are approximately as fast depending on the problem.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Oral)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Machine Vision
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:6402
Deposited By:Tapani Raiko
Deposited On:08 March 2010