PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Online Regret Bounds for Markov Decision Processes with Deterministic Transitions
Ronald Ortner
In: 19th International Conference on Algorithmic Learning Theory, 13-16 Oct 2008, Budapest, Hungary.


We consider an upper confidence bound algorithm for Markov decision processes (MDPs) with deterministic transitions. For this algorithm we derive upper bounds on the online regret (with respect to an (eps-)optimal policy) that are logarithmic in the number of steps taken. These bounds also match known asymptotic bounds for the general MDP setting. We also present corresponding lower bounds. As an application, multi-armed bandits with switching cost are considered.

EPrint Type:Conference or Workshop Item (Talk)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:4582
Deposited By:Ronald Ortner
Deposited On:13 March 2009