PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

SVM with Random Labels
Bruno Apolloni, Simone Bassis and Dario Malchiodi
In: 11th International Conference on Knowledge-Based Intelligent Information and Engineering Systems - XVII Italian Workshop on Neural Networks, 12-14 Sep 2007, Vietri sul Mare (SA), Italy.


We devise an SVM for partitioning a sample space affected by random binary labels. In the hypothesis that a smooth, possibly symmetric, conditional label distribution graduates the passage from the all 0-label domain to the all 1-label domain and under other regularity conditions, the algorithm supplies an estimate of the above probabilities. Within the Algorithmic Inference framework, the randomness of the labels maintains the main features of the binary classification problem, yet adding a further dimension to the search space. Namely the new dimension of each point in the original space hosts the uniform seeds accounting for the randomness of the labels, so that the problem becomes that of separating the points in the augmented space. We solve it with a new kind of bootstrap technique. As for error bounds of the proposed algorithm, we obtain confidence intervals that are up to an order narrower than those supplied in the literature. This benefit comes from the fact that: (i) we devise a special algorithm to take into account the random profile of the labels; (ii) we know the number of support vectors really employed, as an ancillary output of the learning procedure; and (iii) we can appreciate confidence intervals of misclassifying probability exactly in function of the cardinality of these vectors. We numerically check these results by measuring the coverage of the confidence intervals.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:3512
Deposited By:Dario Malchiodi
Deposited On:11 February 2008