Online Regret Bounds for a New Reinforcement Learning Algorithm
Peter Auer and Ronald Ortner
(2005) Working Paper, University of Leoben.

This is the latest version of this eprint.

Abstract

We present a new learning algorithm for undiscounted finite-step reinforcement learning with restarts. Unlike many other analyses in reinforcement learning we are not only interested in convergence properties of the presented UCB algorithm, but aim at bounds for its online performance after a finite number of steps. In the spirit of similar methods already successfully applied for the exploration-exploitation trade-off in multi-armed bandit problems, we use upper confidence bounds to show that the UCB algorithm achieves logarithmic online regret in respect to an optimal policy. The algorithm also performs well in experiments.

Available Versions of this Item

• Online Regret Bounds for a New Reinforcement Learning Algorithm (deposited 28 November 2005)
  • Online Regret Bounds for a New Reinforcement Learning Algorithm (deposited 29 December 2005)
  • Online Regret Bounds for a New Reinforcement Learning Algorithm (deposited 29 December 2005) [Currently Displayed]

[Edit]