PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Taylor-based pseudo-metrics for random process fitting in dynamic programming : expected loss minimisation and risk management
Sylvain Gelly, Jeremie Mary and Olivier Teytaud
In: PDMIA 2005, Lille, France(2005).

Abstract

Stochastic optimization is the research of x optimizing E C (x, A), the expec- tation of C (x, A), where A is a random variable. Typically C (x, a) is the cost related to a strategy x which faces the realization a of the random process. Many stochastic optimization problems deal with multiple time steps, lead- ing to computationally difficult problems ; efficient solutions exist, for example through Bellman’s optimality principle, but only provided that the random process is represented by a well structured process, typically an inhomogeneous Marko- vian process (hopefully with a finite number of states) or a scenario tree. The problem is that in the general case, A is far from being Markovian. So, we look for A , "looking like A", but belonging to a given family A which does not at all contain A. The problem is the numerical evaluation of "A looks like A". A classical method is the use of the Kantorovitch-Rubinstein distance or other transportation metrics [Pflug, 2001], justified by straightforward bounds on the de- viation |E C (x, A) − E C (x, A )| through the use of the Kantorovitch-Rubinstein distance and uniform lipschitz conditions. These approaches might be better than the use of high-level statistics [Keefer, 1994]. We propose other (pseudo- )distances, based upon refined inequalities, guaranteeing a good choice of A . Moreover, as in many cases, we indeed prefer the optimization with risk manage- ment, e.g. optimization of E C (x, noise(A)) where noise(.) is a random noise modelizing the lack of knowledge on the precise random variables, we propose distances which can deal with a user-defined noise. Tests on artificial data sets with realistic loss functions show the relevance of the method.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
ID Code:1545
Deposited By:Sylvain Gelly
Deposited On:28 November 2005