PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A new PAC bound for intersection-closed concept classes
Peter Auer and Ronald Ortner
In: COLT 2004, 1-4 July 2004, Banff, Canada.

There is a more recent version of this eprint available. Click here to view it.

Abstract

For hyper-rectangles in R^d Auer et al. proved a PAC bound of O((d+log 1/delta)/epsilon), where epsilon and delta are the accuracy and confidence parameters. It is still an open question whether one can obtain the same bound for intersection-closed concept classes of VC-dimension d in general. We present a step towards a solution of this problem showing on one hand a new PAC bound of O((d log d + 1/delta)/epsilon) for arbitrary intersection-closed concept classes complementing the well-known bounds O((log 1/delta+d log 1/epsilon)/epsilon) and O(d/epsilon log 1/delta) of Blumer et al. and Haussler et al. Our bound is established using the closure algorithm, that generates as its hypothesis the smallest concept that is consistent with the positive training examples. On the other hand, we show that maximum intersection-closed concept classes meet the bound of O((d+log 1/delta)/epsilon) as well. Moreover, we indicate that our new as well as the conjectured bound cannot hold for arbitrary consistent learning algorithms, giving an example of such an algorithm that needs Omega((d log 1/epsilon + log 1/delta)/epsilon) examples to learn some simple maximum intersection-closed concept class.

EPrint Type:Conference or Workshop Item (Paper)
Additional Information:Available also from http://www.unileoben.ac.at/~infotech/publications/colt2004-ao.pdf
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
ID Code:132
Deposited By:Ronald Ortner
Deposited On:25 December 2004

Available Versions of this Item