Efficient Bounds for the softmax Function, Applications to Inference in Hybrid Models
Guillaume Bouchard
In: NIPS 2007, 02-08 December 2007, Whistler, British Columbia, Canada.

Abstract

We introduce the problem of variational inference in probabilistic model where a multinomial variable is defined conditionally to continous parents through a softmax function. Efficient Bayesian inference for this type of model is still an open problem due to the lack of efficient upper bound for the sum of exponentials. We propose three different bounds and compare them on simulations. As a real world example, the classification performances of the proposed bounding techniques for the Bayesian multinomial logistic regression are compared to approximations based on sampling. We finally discuss the pros and cons of "bounding" versus "approximating" the true posterior distribution.

PDF (Technical report) - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]