PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Adaptive aggregation for Reinforcement Learning with efficient exploration: deterministic domains
Andrey Bernstein and Nahum Shimkin
In: The 21st Annual Conference on Learning Theory (COLT 2008), 9-12 Jul 2008, Helsinki, Finland.


We propose a model-based learning algorithm, the Adaptive Aggregation Algorithm (AAA), that aims to solve the online, continuous state space reinforcement learning problem in a deterministic domain. The proposed algorithm uses an adaptive state aggregation approach, going from coarse to fine grids over the state space, which enables to use finer resolution in the “important” areas of the state space, and coarser resolution elsewhere. We consider an on-line learning approach, in which we discover these important areas on-line, using an uncertainty intervals exploration technique. Polynomial learning rates in terms of mistake bound (in a PAC framework) are established for this algorithm, under appropriate continuity assumptions.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:5854
Deposited By:Andrey Bernstein
Deposited On:08 March 2010