Inadequacy of interval estimates corresponding to variational Bayesian approximations
Bo Wang and Mike Titterington
In: 10th International workshop on Artifical Intelligence and Statistics, 6-8 Jan 2005, Barbados.
In this paper we investigate the properties of the covariance matrices associated with variational Bayesian approximations, based on data from mixture models, and compare them with the true covariance matrices, corresponding to Fisher information matrices. It is shown that the covariance matrices from the variational Bayes approximations are normally `too small' compared with those for the maximum likelihood estimator, so that resulting interval estimates for the parameters will be unrealistically narrow, especially if the components of the mixture model are not well separated.