Learning Natural Image Structure with a Horizonal Product Model
Urs Köster, Jussi T. Lindgren, Michael Gutmann and Aapo Hyvärinen
In: ICA 2009, 15-18 Mar 2009, Paraty RJ / Brazil.
We present a novel extension to Independent Component
Analysis (ICA), where the data is generated as the product of two submodels,
each of which follow an ICA model, and which combine in a
horizontal fashion. This is in contrast to previous nonlinear extensions
to ICA which were based on a hierarchy of layers. We apply the product
model to natural image patches and report the emergence of localized
masks in the additional network layer, while the Gabor features that are
obtained in the primary layer change their tuning properties and become
less localized. As an interpretation we suggest that the model learns to
separate the localiztion of image features from other properties, since
identity and position of a feature are plausibly independent. We also
show that the horizontal model can be interpreted as an overcomplete
model where the features are no longer indepedent.