Importance of Cross-Layer Cooperation for Learning Deep Feature Hierarchies
Gregoire Montavon, Mikio braun and Klaus-Robert Müller
In: A common property of hierarchical models of the brain is their capacity to in- tegrate bottom-up and top-down information in order to distill the task-relevant information from the sensory noise. In this paper, we argue that such coopera- tion between upp, 16 Dec 2011, Granda, Spain.

Abstract

A common property of hierarchical models of the brain is their capacity to integrate bottom-up and top-down information in order to distill the task-relevant information from the sensory noise. In this paper, we argue that such cooperation between upper and lower layers is not only useful at prediction time but also at learning time in order to build a successful feature hierarchy. The claim is corroborated by training a set of deep networks on real data and measuring the evolution of the representation layer after layer. The analysis reveals that cross-layer cooperation enables the emergence of a sequence of increasingly invariant representations.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]