Statistical Properties of Kernel Principal Component Analysis
Olivier Bousquet, Laurent Zwald and Gilles Blanchard
In: 17th. Conference on Learning Theory (COLT 2004), 1-4 July 2004, Banff, Canada.
We study the properties of the eigenvalues of Gram matrices in a
non-asymptotic setting. Using local Rademacher averages, we
provide data-dependent and tight bounds for their convergence
towards eigenvalues of the corresponding kernel operator. We
perform these computations in a functional analytic framework
which allows to deal implicitly with reproducing kernel Hilbert
spaces of infinite dimension. This can have applications to
various kernel algorithms, such as Support Vector Machines (SVM).
We focus on Kernel Principal Component Analysis (KPCA) and, using
such techniques, we obtain sharp excess risk bounds for the
reconstruction error. In these bounds, the dependence on the decay
of the spectrum and on the closeness of successive eigenvalues is