PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Fitted Q-iteration in continuous action-space MDPs
András Antos, Rémi Munos and Csaba Szepesvari
In: Advances in Neural Information Processing Systems (2008) MIT Press , Cambridge, MA, USA , pp. 9-16.


We consider continuous state, continuous action batch reinforcement learning where the goal is to learn a good policy from a sufficiently rich trajectory generated by some policy. We study a variant of fitted Q-iteration, where the greedy action selection is replaced by searching for a policy in a restricted set of candidate policies by maximizing the average action-values. We provide a rigorous analysis of this algorithm, proving what we believe is the first finite-time bound for value-function based algorithms for continuous state and action problems.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
Postscript - PASCAL Members only - Requires a viewer, such as GhostView
EPrint Type:Book Section
Additional Information:Proceedings: Twenty-First Annual Conference on Neural Information Processing Systems, NIPS 2007, poster, paper no. 917, Vancouver, B.C., December 3-6, 2007.
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:3804
Deposited By:András Antos
Deposited On:09 February 2008