A Tutorial on Kernel Methods for Categorization
F. Jäkel, B. Schölkopf and F.A. Wichmann
Journal of Mathematical Psychology
The abilities to learn and to categorize are fundamental for cognitive 8 systems, be it animals or machines, and therefore have attracted attention 9 from engineers and psychologists alike. Modern machine learning meth- 10 ods and psychological models of categorization are remarkably similar, 11 partly because these two fields share a common history in artificial neural 12 networks and reinforcement learning. However, machine learning is now 13 an independent and mature field that has moved beyond psychologically 14 or neurally inspired algorithms towards providing foundations for a theory 15 of learning that is rooted in statistics and functional analysis. Much of 16 this research is potentially interesting for psychological theories of learn- 17 ing and categorization but also hardly accessible for psychologists. Here, 18 we provide a tutorial introduction to a popular class of machine learn- 19 ing tools, called kernel methods. These methods are closely related to 20 perceptrons, radial-basis-function neural networks and exemplar theories 21 of categorization. Recent theoretical advances in machine learning are 22 closely tied to the idea that the similarity of patterns can be encapsulated 23 in a positive definite kernel. Such a positive definite kernel can define a 24 reproducing kernel Hilbert space which allows one to use powerful tools 25 from functional analysis for the analysis of learning algorithms. We give 26 basic explanations of some key concepts—the so-called kernel trick, the 27 representer theorem and regularization—which may open up the possibil- 28 ity that insights from machine learning can feed back into psychology.