PAC-Bayesian Generalization Bound for Density Estimation with Application to Co-clustering.
Yevgeny Seldin and Naftali Tishby
In proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AIStats 2009) 2009.

Abstract

We derive a PAC-Bayesian generalization bound for density estimation. Similar to the PAC-Bayesian generalization bound for classi cation, the result has the appealingly simple form of a tradeo between empirical performance and the KL-divergence of the posterior from the prior. Moreover, the PACBayesian generalization bound for classication can be derived as a special case of the bound for density estimation. To illustrate a possible application of our bound we derive a generalization bound for co-clustering. The bound provides a criterion to evaluate the ability of co-clustering to predict new co-occurrences, thus introducing the notion of generalization to this traditionally unsupervised task.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]