PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Online Regret Bounds for Markov Decision Processes with Deterministic Transitions
Ronald Ortner
In: Proceedings of the 19th International Conference on Algorithmic Learning Theory Lecture Notes in Artificial Intelligence (5254). (2008) Springer , pp. 123-137.


We consider an upper confidence bound algorithm for Markov decision processes (MDPs) with deterministic transitions. For this algorithm we derive upper bounds on the online regret (with respect to an (eps-)optimal policy) that are logarithmic in the number of steps taken. These bounds also match known asymptotic bounds for the general MDP setting. We also present corresponding lower bounds. As an application, multi-armed bandits with switching cost are considered.

EPrint Type:Book Section
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:4581
Deposited By:Ronald Ortner
Deposited On:13 March 2009