Construction of Nonparametric Bayesian Models from Parametric Bayes Equations
In: NIPS 2009, 6-9 Dec 2009, Vancouver, BC, Canada.
We consider the general problem of constructing nonparametric
Bayesian models on infinite-dimensional random objects, such
as functions, infinite graphs or infinite permutations.
The problem has generated much interest in machine learning,
where it is treated heuristically, but has not been
studied in full generality in nonparametric Bayesian statistics, which tends to
focus on models over probability distributions.
Our approach applies a standard tool of stochastic process
theory, the construction of stochastic processes from their
finite-dimensional marginal distributions.
The main contribution of the paper is a generalization
of the classic Kolmogorov extension theorem to conditional
This extension allows a rigorous construction of nonparametric Bayesian models
from systems of finite-dimensional, parametric Bayes equations.
Using this approach, we show (i)
how existence of a conjugate posterior for
the nonparametric model can be guaranteed by choosing
conjugate finite-dimensional models in the construction, (ii) how the
mapping to the posterior parameters of the nonparametric
model can be explicitly determined, and (iii) that
the construction of conjugate models in essence requires the
finite-dimensional models to be in the exponential family.
As an application of our constructive framework,
we derive a model on infinite
permutations, the nonparametric Bayesian analogue of a model
recently proposed for the analysis of rank data.
|EPrint Type:||Conference or Workshop Item (Paper)|
|Project Keyword:||Project Keyword UNSPECIFIED|
|Subjects:||Theory & Algorithms|
|Deposited By:||Peter Orbanz|
|Deposited On:||08 March 2010|