A Gradient-Based Algorithm Competitive with Variational Bayesian EM for Mixture of Gaussians
Mikael Kuusela, Tapani Raiko, Antti Honkela and Juha Karhunen
In: International Joint Conference on Neural Networks, 14-19 June 2009, Atlanta, USA.
While variational Bayesian (VB) inference is typically
done with the so called VB EM algorithm, there are
models where it cannot be applied because either the E-step or
the M-step cannot be solved analytically. In 2007, Honkela et
al. introduced a recipe for a gradient-based algorithm for VB
inference that does not have such a restriction. In this paper,
we derive the algorithm in the case of the mixture of Gaussians
model. For the first time, the algorithm is experimentally
compared to VB EM and its variant with both artificial and
real data. We conclude that the algorithms are approximately
as fast depending on the problem.