PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Efficient Bounds for the softmax Function, Applications to Inference in Hybrid Models
Guillaume Bouchard
In: NIPS 2007, 02-08 December 2007, Whistler, British Columbia, Canada.


We introduce the problem of variational inference in probabilistic model where a multinomial variable is defined conditionally to continous parents through a softmax function. Efficient Bayesian inference for this type of model is still an open problem due to the lack of efficient upper bound for the sum of exponentials. We propose three different bounds and compare them on simulations. As a real world example, the classification performances of the proposed bounding techniques for the Bayesian multinomial logistic regression are compared to approximations based on sampling. We finally discuss the pros and cons of "bounding" versus "approximating" the true posterior distribution.

PDF (Technical report) - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Oral)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:3498
Deposited By:Guillaume Bouchard
Deposited On:11 February 2008