PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-clustering
Yevgeny Seldin and Naftali Tishby
In: AISTATS 2009, 16-18 Apr 2009, Clearwater Beach, Florida USA.

Abstract

We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian generalization bound for classification, the result has the appealingly simple form of a trade-off between empirical performance and the KL-divergence of the posterior from the prior. Moreover, the PAC-Bayesian generalization bound for classification can be derived as a special case of the bound for density estimation. To illustrate a possible application of our bound we derive a generalization bound for co-clustering. The bound provides a criterion to evaluate the ability of co-clustering to predict new co-occurrences, thus introducing the notion of generalization to this traditionally unsupervised task.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]