PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Active learning with heteroscedastic noise
András Antos, Varun Grover and Csaba Szepesvari
Theoretical Computer Science Volume 411, Number 29-30, pp. 2712-2728, 2010. ISSN 0304-3975

Abstract

We consider the problem of actively learning the mean values of distributions associated with a finite number of options. The decision maker can select which option to generate the next observation from, the goal being to produce estimates with equally good precision for all the options. If sample means are used to estimate the unknown values then the optimal solution, assuming that the distributions are known up to a shift, is to sample from each distribution proportional to its variance. No information other than the distributions’ variances is needed to calculate the optimal solution. In this paper we propose an incremental algorithm that asymptotically achieves the same loss as an optimal rule. We prove that the excess loss suffered by this algorithm, apart from logarithmic factors, scales as n−{3/2}, which we conjecture to be the optimal rate. The performance of the algorithm is illustrated on a simple problem.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Additional Information:Special Issue for ALT 2008. Eds.: Y. Freund, L. Györfi, Gy. Turán, T. Zeugmann
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:4761
Deposited By:András Antos
Deposited On:16 March 2009