Learning by mirror averaging
A. Juditsky, Ph. Rigollet and A. Tsybakov
Annals of Statistics Volume 36, Number 5, pp. 2183-2206, 2008.

Abstract

Given a finite collection of estimators or classi¯ers, we study the problem of model selection type aggregation, i.e., we construct a new estimator or classi¯er, called aggregate, which is nearly as good as the best among them with respect to a given risk criterion. We de¯ne our aggregate by a simple recursive procedure which solves an auxiliary stochastic linear programming problem related to the original non-linear one and constitutes a special case of the mirror averaging algorithm. We show that the aggregate satisfies sharp oracle inequalities under some general assumptions. The results are applied to several problems including regression, classification and density estimation.

[Edit]