PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Selecting the state-representation in reinforcement learning
Odalric Maillard, Rémi Munos and Daniil Ryabko
NIPS 2011 2011.


The problem of selecting the right state-representation in a reinforcement learning problem is considered. Several models (functions mapping past observations to a finite set) of the observations are given, and it is known that for at least one of these models the resulting state dynamics are indeed Markovian. Without know ing neither which of the models is the correct one, nor what are the probabilistic characteristics of the resulting MDP, it is required to obtain as much reward as the optimal policy for the correct model (or for the best of the correct models, if there are several). We propose an algorithm that achieves that, with a regret of order T2/3 where T is the horizon time.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
ID Code:8985
Deposited By:Rémi Munos
Deposited On:21 February 2012