PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Online Classification for Complex Problems Using Simultaneous Projections
Yonatan Amit, Shai Shalev-Shwartz and Yoram Singer
JMLR 2008.


We describe and analyze an algorithmic framework for online classification where each online trial consists of {\em multiple} prediction tasks that are tied together. We tackle the problem of updating the online hypothesis by defining a projection problem in which each prediction task corresponds to a single linear constraint. These constraints are tied together through a single slack parameter. We then introduce a general method for approximately solving the problem by projecting {\em simultaneously} and independently on each constraint which corresponds to a prediction sub-problem, and then averaging the individual solutions. We show that this approach constitutes a feasible, albeit not necessarily optimal, solution for the original projection problem. We derive concrete simultaneous projection schemes and analyze them in the mistake bound model. We demonstrate the power of the proposed algorithm in experiments with online multiclass text categorization. Our experiments indicate that a combination of class-dependent features with the simultaneous projection method outperforms previously studied algorithms.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:4065
Deposited By:Shai Shalev-Shwartz
Deposited On:25 February 2008