PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Datum-wise classification. A sequential Approach to sparsity
Gabriel Dulac-Arnold, Ludovic Denoyer, Philippe Preux and Patrick Gallinari
In: ECML/PKDD 2011, 5-9 Sep, 2011, Athens, Greece.

Abstract

We propose a novel classification technique whose aim is to select an appropriate representation for each datapoint, in contrast to the usual approach of selecting a representation encompassing the whole dataset. This datum-wise representation is found by using a sparsity inducing empirical risk, which is a relaxation of the standard L0 regular- ized risk. The classification problem is modeled as a sequential decision process that sequentially chooses, for each datapoint, which features to use before classifying. Datum-Wise Classification extends naturally to multi-class tasks, and we describe a specific case where our inference has equivalent complexity to a traditional linear classifier, while still using a variable number of features. We compare our classifier to classical L1 regularized linear models (L1 -SVM and LARS) on a set of common bi- nary and multi-class datasets and show that for an equal average number of features used we can get improved performance using our method.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
ID Code:8290
Deposited By:Philippe Preux
Deposited On:29 July 2011