PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Performances of some learning algorithms: Kernel Projection Machine and kernel principal component analysis
Laurent Zwald
(2005) PhD thesis, University Paris XI (Orsay).


This thesis takes place within the framework of statistical learning. It brings contributions to the machine learning community using modern statistical techniques based on progress in the study of empirical processes. The first part investigates the statistical properties of Kernel Principal Component Analysis (KPCA). The behavior of the reconstruction error is studied with a non-asymptotic point of view and concentration inequalities of the eigenvalues of the kernel matrix are provided. All these results correspond to fast convergence rates. Non-asymptotic results concerning the eigenspaces of KPCA themselves are also provided. A new algorithm of classification has been designed in the second part: the Kernel Projection Machine (KPM). It is inspired by the Support Vector Machines (SVM). Besides, it highlights that the selection of a vector space by a dimensionality reduction method such as KPCA regularizes suitably. The choice of the vector space involved in the KPM is guided by statistical studies of model selection using the penalized minimization of the empirical loss. This regularization procedure is intimately connected with the finite dimensional projections studied in the statistical work of Birg\'e and Massart. The performances of KPM and SVM are then compared on some data sets. Each topic tackled in this thesis raises new questions.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Thesis (PhD)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:1854
Deposited By:Laurent Zwald
Deposited On:29 November 2005