PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Error Bounds for Approximations from Projected Linear Equations
Huizhen Yu and Dimitri Bertsekas
Mathematics of Operations Research 2008.


We consider linear fixed point equations and their approximations by projection on a low dimensional subspace. We derive new bounds on the approximation error of the solution, which are expressed in terms of low dimensional matrices and can be computed by simulation. When the fixed point mapping is a contraction, as is typically the case in Markov decision processes (MDP), one of our bounds is always sharper than the standard contraction-based bounds, and another one is often sharper. The former bound is also tight in a worst-case sense. Our bounds also apply to the non-contraction case, including policy evaluation in MDP with nonstandard projections that enhance exploration. There are no error bounds currently available for this case to our knowledge.

EPrint Type:Article
Additional Information:The technical report version: A shorter version appeared at European Workshop on Reinforcement Learning (EWRL), Lille, France, 2008.
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:4658
Deposited By:Huizhen Yu
Deposited On:13 March 2009