PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A Class of Rényi Information Estimators for Multidimensional Densities
Nikolai Leonenko, Luc Pronzato and Vippal Savani
Annals of Statistics 2005.


A class of estimators of the R\'enyi and Tsallis entropies of an unknown distribution $f$ in $\mathbb{R}^m$ is presented, based on $k$-th nearest-neighbor distances in a sample of $N$ i.i.d.\ vectors distributed with $f$. We show that entropies of any order $q$ can be estimated consistently with minimal assumptions on $f$. The method can be extended straightforwardly to the estimation of the statistical distance between two distributions using one i.i.d.\ sample from each.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:1031
Deposited By:Luc Pronzato
Deposited On:22 July 2005