|
Approximated geodesic updates with principal natural gradients
Zhirong Yang and Jorma Laaksonen
In: The 2007 International Joint Conference on Neural Networks (IJCNN 2007), 12-17 Aug 2007, Orlando, USA.
AbstractWe propose a novel optimization algorithm which
overcomes two drawbacks of Amari’s natural gradient updates
for information geometry. First, prewhitening the tangent vectors
locally converts a Riemannian manifold to an Euclidean
space so that the additive parameter update sequence approximates
geodesics. Second, we prove that dimensionality reduction
of natural gradients is necessary for learning multidimensional
linear transformations. Removal of minor components also
leads to noise reduction and better computational efficiency.
The proposed method demonstrates faster and more robust convergence
in the simulations on recovering a Gaussian mixture
of artificial data and on discriminative learning of ionosphere
data. [Edit]
|
