PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Chromatic PAC-Bayes Bounds for Non-IID Data
Liva Ralaivola, Marie Szafranski and Guillaume Stempfel
In: AISTATS 09, 16-18 Apr 2009, Clearwater.

Abstract

\pac-Bayes bounds are among the most accurate generalization bounds for classifiers learned with \iid data, and it is particularly so for margin classifiers. However, there are many practical cases where the training data show some dependencies and where the traditional \iid assumption does not apply. Stating generalization bounds for such frameworks is therefore of the utmost interest, both from theoretical and practical standpoints. In this work, we propose the first --~to the best of our knowledge~-- \pac-Bayes generalization bounds for classifiers trained on data exhibiting interdependencies. The approach undertaken to establish our results is based on the decomposition of a so-called dependency graph that encodes the dependencies within the data, in sets of independent data, through the tool of graph fractional covers. Our bounds are very general, since being able to find an upper bound on the (fractional) chromatic number of the dependency graph is sufficient to get new \pac-Bayes bounds for specific settings. We show how our results can be used to derive bounds for bipartite ranking and windowed prediction on sequential data.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:5032
Deposited By:Liva Ralaivola
Deposited On:24 March 2009