A Comparison of Tight Generalization Error Bounds
Matti Kääriäinen and John Langford
In: ICML 2005, 7-11 Aug 2005, Bonn, Germany.
We investigate the empirical applicability of several bounds (a number of
which are new) on the true error rate of learned classifiers which hold
whenever the examples are chosen independently at random from a fixed
The collection of tricks we use includes:
1. A technique using unlabeled data for a tight derandomization of
2. A tight form of the progressive validation bound.
3. The exact form of the test set bound.
The bounds are implemented in the semibound package and are