Bolasso: model consistent Lasso estimation through the bootstrap.
In: ICML 2008, Helsinki, Finland(2008).
We consider the least-square linear regression
problem with regularization by the ℓ1-norm, a
problem usually referred to as the Lasso. In this
paper, we present a detailed asymptotic analysis
of model consistency of the Lasso. For various
decays of the regularization parameter, we
compute asymptotic equivalents of the probability
of correct model selection (i.e., variable selection).
For a specific rate decay, we show that the
Lasso selects all the variables that should enter
the model with probability tending to one exponentially
fast, while it selects all other variables
with strictly positive probability. We show that
this property implies that if we run the Lasso for
several bootstrapped replications of a given sample,
then intersecting the supports of the Lasso
bootstrap estimates leads to consistent model selection.
This novel variable selection algorithm,
referred to as the Bolasso, is compared favorably
to other linear regression methods on synthetic
data and datasets from the UCI machine learning
|EPrint Type:||Conference or Workshop Item (Talk)|
|Project Keyword:||Project Keyword UNSPECIFIED|
|Subjects:||Theory & Algorithms|
|Deposited By:||Francis Bach|
|Deposited On:||13 March 2009|