PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Training parsers by inverse reinforcement learning
G Neu and Csaba Szepesvari
Journal of Machine Learning Volume 77, pp. 303-337, 2009. ISSN http://www.sztaki.hu/~szcsaba/papers/MLJ-SISP-09.pdf

Abstract

One major idea in structured prediction is to assume that the predictor computes its output by nding the maximum of a score function. The training of such a predictor can then be cast as the problem of nding weights of the score function so that the output of the predictor on the inputs matches the corresponding structured labels on the training set. A similar problem is studied in inverse reinforcement learning (IRL) where one is given an environment and a set of trajectories and the problem is to nd a reward function such that an agent acting optimally with respect to the reward function would follow trajectories that match those in the training set. In this paper we show how IRL algorithms can be applied to structured prediction, in particular to parser training. We present a number of recent incremental IRL algorithms in a unied framework and map them to parser training algorithms. This allows us to recover some existing parser training algorithms, as well as to obtain a new one. The resulting algorithms are compared in terms of their sensitivity to the choice of various parameters and generalization ability on the Penn Treebank WSJ corpus.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Natural Language Processing
Theory & Algorithms
ID Code:6345
Deposited By:Csaba Szepesvari
Deposited On:08 March 2010