PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Linear and convex aggregation of density estimators.
Philippe Rigollet and Alexandre Tsybakov
Mathematical Methods of Statistics Volume 16, Number 3, pp. 260-280, 2007.


We study the problem of learning the best linear and convex combination of $M$ estimators of a density with respect to the mean squared risk. We suggest aggregation procedures and we prove sharp oracle inequalities for their risks, i.e., oracle inequalities with leading constant 1. We also obtain lower bounds showing that these procedures attain optimal rates of aggregation. As an example, we consider aggregation of multivariate kernel density estimators with different bandwidths. We show that linear and convex aggregates mimic the kernel oracles in asymptotically exact sense. We prove that, for Pinsker's kernel, the proposed aggregates are sharp asymptotically minimax simultaneously over a large scale of Sobolev classes of densities. Finally, we provide simulations demonstrating performance of the convex aggregation procedure.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:3862
Deposited By:Alexandre Tsybakov
Deposited On:25 February 2008