State-of-the-Art and Evolution in Public Data Sets and Competitions for System Identification, Time Series Prediction and Pattern Recognition
Joos Vandewalle, Johan Suykens, Bart De Moor and Amaury Lendasse
In: Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on, 15-20 April 2007, Honolulu, HI.
It is the aim of reproducible research to provide mechanisms for objective comparison of methods, algorithms, software and procedures in various research topics. In this paper, we discuss the role of data sets, benchmarks and competitions in the fields of system identification, time series prediction, classification, and pattern recognition in view of creating an environment of reproducible research. Important elements are the data sets, their origin, and the comparison measures that will be used to rank the performance of the methods. The issues are discussed, a comparison is made and recommendations are given