An Auxiliary Variational Method
Felix Agakov and David Barber
In: ICONIP 2004, 22-25 Nov 2004, Calcutta, India.
An attractive feature of variational methods used in the context of approximate inference in undirected graphical models is a rigorous lower bound on the normalization constants. Here we explore the idea of using augmented variable spaces to improve on the standard mean-field bounds. Our approach forms a more powerful class of approximations than any structured mean field technique. Moreover, the existing variational mixture models may be seen as computationally expensive special cases of our method. A byproduct of our work is an efficient way to calculate a set of mixture coefficients for any set of tractable distributions that principally improves on a flat combination.