PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Online Classification for Complex Problems Using Simultaneous Projections
Yonatan Amit, Shai Shalev-Shwartz and Yoram Singer
In: NIPS 2006(2006).


We describe and analyze an algorithmic framework for online classification where each online trial consists of {\em multiple} prediction tasks that are tied together. We tackle the problem by defining an instantaneous projection problem in which all the prediction tasks are tied through a single slack parameter. We then introduce a general method for approximately solving the problem by projecting {\em simultaneously} and independently each constraint which corresponds to a prediction sub-problem, and then average the individual solutions. We show that this approach constitutes a feasible, albeit not necessarily optimal, solution for the original projection problem. We derive concrete simultaneous projection variants and analyze them in the mistake bound model. We demonstrate the power of the proposed algorithm in experiments with online multiclass text categorization. Our experiments indicate that a combination of class-dependent features with the simultaneous projection method outperforms previous algorithms for this task.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Natural Language Processing
Theory & Algorithms
ID Code:2246
Deposited By:Shai Shalev-Shwartz
Deposited On:10 October 2006