PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Approximated geodesic updates with principal natural gradients
Zhirong Yang and Jorma Laaksonen
In: The 2007 International Joint Conference on Neural Networks (IJCNN 2007), 12-17 Aug 2007, Orlando, USA.


We propose a novel optimization algorithm which overcomes two drawbacks of Amari’s natural gradient updates for information geometry. First, prewhitening the tangent vectors locally converts a Riemannian manifold to an Euclidean space so that the additive parameter update sequence approximates geodesics. Second, we prove that dimensionality reduction of natural gradients is necessary for learning multidimensional linear transformations. Removal of minor components also leads to noise reduction and better computational efficiency. The proposed method demonstrates faster and more robust convergence in the simulations on recovering a Gaussian mixture of artificial data and on discriminative learning of ionosphere data.

EPrint Type:Conference or Workshop Item (Oral)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:3648
Deposited By:Jorma Laaksonen
Deposited On:14 February 2008