A faster model selection criterion for OP-ELM and OP-KNN: HQ criterion
Yoan Miche and Amaury Lendasse
In: ESANN'09 Conference, 22 - 24 April 2009, Bruges, Belgium.
The Optimally Pruned Extreme Learning Machine (OPELM)
and Optimally Pruned K-Nearest Neighbors (OP-KNN) algorithms
use the a similar methodology based on random initialization (OP-ELM)
or KNN initialization (OP-KNN) of a Feedforward Neural Network followed
by ranking of the neurons; ranking is used to determine the best
combination to retain. This is achieved by Leave-One-Out (LOO) crossvalidation.
In this article is proposed to use the Hannan-Quinn (HQ)
Criterion as a model selection criterion, instead of LOO. It proved to be
efficient and as good as the LOO one for both OP-ELM and OP-KNN,
while decreasing computations by factors of four to five for OP-ELM and
up to 24 for OP-KNN.