PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A new PAC bound for intersection-closed concept classes
Peter Auer and Ronald Ortner
Machine Learning Volume 66, pp. 151-163, 2006.

This is the latest version of this eprint.


For hyper-rectangles in R^d Auer (1997) proved a PAC bound of O((d + log1/delta)/epsilon), where epsilon and delta are the accuracy and confidence parameters. It is still an open question whether one can obtain the same bound for intersection-closed concept classes of VC-dimension d in general. We present a step towards a solution of this problem showing on one hand a new PAC bound of O((d log d + log 1/delta)/epsilon) for arbitrary intersection-closed concept classes, complementing the well-known bounds O((log 1/delta+d log 1/epsilon)/epsilon) and O(d/epsilon log 1/delta) of Blumer et al. and (1989) and Haussler, Littlestone and Warmuth (1994). Our bound is established using the closure algorithm, that generates as its hypothesis the intersection of all concepts that are consistent with the positive training examples. On the other hand, we show that many intersection-closed concept classes including e.g. maximum intersection-closed classes satisfy an additional combinatorial property that allows a proof of the optimal bound of O((d + log1/delta)/epsilon). For such improved bounds the choice of the learning algorithm is crucial, as there are consistent learning algorithms that need Omega((d log 1/epsilon + log1/delta)/epsilon) examples to learn some particular maximum intersection-closed concept classes.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:1330
Deposited By:Ronald Ortner
Deposited On:28 November 2005

Available Versions of this Item