PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Efficient continuous-time reinforcement learning with adaptive state graphs
Gerhard Neumann, Michael Pfeiffer and Wolfgang Maass
In: ECML/PKDD 2007, 17 Sep - 21 Sep 2007, Warsaw, Poland.


We present a new reinforcement learning approach for deterministic continuous control problems in environments with unknown, arbitrary reward functions. The difficulty of finding solution trajectories for such problems can be reduced by incorporating limited prior knowledge of the approximative local system dynamics. The presented algorithm builds an adaptive state graph of sample points within the continuous state space. The nodes of the graph are generated by an efficient principled exploration scheme that directs the agent towards promising regions, while maintaining good online performance. Global solution trajectories are formed as combinations of local controllers that connect nodes of the graph, thereby naturally allowing continuous actions and continuous time steps. We demonstrate our approach on various movement planning tasks in continuous domains.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:3076
Deposited By:Michael Pfeiffer
Deposited On:07 December 2007