V-fold penalization: an alternative to V-fold cross-validation
Mathematisches Forschungsinstitut Oberwolfach Report
European Mathematical Society
One of the most widely used model selection techniques is V-fold cross-validation. We study some of its properties from the non-asymptotic viewpoint, in particular with the goal of choosing the optimal V.
Then, following Efron's resampling heuristics (Efron, 1979), we propose to use a V-fold resampling scheme to define a new penalization procedure, called V-fold penalization. In a regression framework with heteroscedastic noise, we prove a non-asymptotic oracle inequality with constant almost one, implying asymptotic optimality.
A simulation study finally shows that V-fold penalties outperform V-fold cross-validation and Mallows' penalties in several heteroscedastic cases.