PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

LSTD with Random Projections
Mohammad Ghavamzadeh, Alessandro Lazaric, Odalric-Ambrym Maillard and Rémi Munos
In: Twenty-Fourth Annual Conference on Advances in Neural Information Processing Systems (NIPS-2010), 6-9 Dec 2010, Vancouver, Canada.


We consider the problem of reinforcement learning in high-dimensional spaces when the number of features is bigger than the number of samples. In particular, we study the least-squares temporal difference (LSTD) learning algorithm when a space of low dimension is generated with a random projection from a high-dimensional space. We provide a thorough theoretical analysis of the LSTD with random projections and derive performance bounds for the resulting algorithm. We also show how the error of LSTD with random projections is propagated through the iterations of a policy iteration algorithm and provide a performance bound for the resulting least-squares policy iteration (LSPI) algorithm.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:7384
Deposited By:Mohammad Ghavamzadeh
Deposited On:17 March 2011