PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

The Power of Selective Memory: Self-Bounded Learning of Prediction Suffix Trees
Ofer Dekel, Shai Shalev-Shwartz and Yoram Singer
In: NIPS 2004, December 13-16, 2004, Vancouver and Whistler, British Columbia, Canada.


Prediction suffix trees (PST) provide a popular and effective tool for tasks such as compression, classification, and language modeling. In this paper we take a decision theoretic view of PSTs. Generalizing the notion of margin to PSTs, we present an online PST learning algorithm and derive a mistake bound for it. We then describe a self-bounded enhancement of our learning algorithm for which the learning process automatically grows a {\em bounded-depth} PST. We also prove a similar mistake-bound for the self-bounded algorithm. The result is an efficient algorithm that neither relies on a-priori assumptions on the shape or maximal depth of the target PST nor does it require any parameters. To our knowledge, this is the first provably-correct PST learning algorithm which generates a bounded-depth PST while being competitive with any fixed PST determined in hindsight.

Postscript - Requires a viewer, such as GhostView
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:423
Deposited By:Shai Shalev-Shwartz
Deposited On:21 December 2004