A statistical physics approach for the analysis of machine learning
algorithms on real data.
Manfred Opper and Manfred Opper
Journal of Statistical Mechanics
We combine the replica approach of statistical physics with a
variational technique to make it applicable for the analysis of
machine learning algorithms on real data. The method is applied to
Gaussian process models and their relative, the Support Vector
machine. We discuss the quality of our theoretical results in
comparison to experiments. As a key result, we apply our theory on
real world benchmark data and show its potential for practical
applications by deriving approximate expressions for data averaged
performance measures which hold for general data distributions and
allow to optimize the performance of the learning algorithm.