PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Kernel Analysis of Deep Networks
Gregoire Montavon, Mikio braun and Klaus-Robert Müller
Journal of Machine Learning Research (JMLR) Volume 12, pp. 2563-2581, 2011. ISSN 1533-7928


When training deep networks it is common knowledge that an efficient and well generalizing representation of the problem is formed. In this paper we aim to elucidate what makes the emerging representation successful. We analyze the layer-wise evolution of the representation in a deep network by building a sequence of deeper and deeper kernels that subsume the mapping performed by more and more layers of the deep network and measuring how these increasingly complex kernels fit the learning problem. We observe that deep networks create increasingly better representations of the learning problem and that the structure of the deep network controls how fast the representation of the task is formed layer after layer.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:9497
Deposited By:Mikio braun
Deposited On:16 March 2012