PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-clustering
Yevgeny Seldin and Naftali Tishby
In: AISTATS 2009, 16-18 Apr 2009, Clearwater Beach, Florida USA.

Abstract

We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian generalization bound for classification, the result has the appealingly simple form of a trade-off between empirical performance and the KL-divergence of the posterior from the prior. Moreover, the PAC-Bayesian generalization bound for classification can be derived as a special case of the bound for density estimation. To illustrate a possible application of our bound we derive a generalization bound for co-clustering. The bound provides a criterion to evaluate the ability of co-clustering to predict new co-occurrences, thus introducing the notion of generalization to this traditionally unsupervised task.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:5959
Deposited By:Yevgeny Seldin
Deposited On:08 March 2010