PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Penalized Partial Least Squares with Applications to B-Spline Transformations and Functional Data
Nicole Krämer, Anne-Laure Boulesteix and Gerhard Tutz
Chemometrics & Intelligent Laboratory Systems Volume 94, Number 1, pp. 60-69, 2008.

Abstract

We propose a novel framework that combines penalization techniques with Partial Least Squares (PLS). We focus on two important applications. (1) We combine PLS with a roughness penalty to estimate high-dimensional regression problems with functional predictors and scalar response. (2) Starting with an additive model, we expand each variable in terms of a generous number of B-spline basis functions. To prevent overfitting, we estimate the model by applying a penalized version of PLS. We gain additional model flexibility by incorporating a sparsity penalty. Both applications can be formulated in terms of a unified algorithm called Penalized Partial Least Squares, which can be computed virtually as fast as PLS using the kernel trick. Furthermore, we prove a close connection of penalized PLS to preconditioned linear systems. In experiments, we show the benefits of our method to noisy functional data and to sparse nonlinear regression models.

EPrint Type:Article
Additional Information:A preprint is available at http://ml.cs.tu-berlin.de/~nkraemer/publications.html
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:4176
Deposited By:Nicole Krämer
Deposited On:18 October 2008