PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Improved Learning of Gaussian-Bernoulli Restricted Boltzmann Machines
KyungHyun Cho, Alexander Ilin and Tapani Raiko
In: International Conference on Artificial Neural Networks (ICANN 2011), 14-17 Jun 2011, Espoo, Finland.


We propose a few remedies to improve training of Gaussian-Bernoulli restricted Boltzmann machines (GBRBM), which is known to be difficult. Firstly, we use a different parameterization of the energy function, which allows for more intuitive interpretation of the parameters and facilitates learning. Secondly, we propose parallel tempering learning for GBRBM. Lastly, we use an adaptive learning rate which is selected automatically in order to stabilize training. Our extensive experiments show that the proposed improvements indeed remove most of the difficulties encountered when training GBRBMs using conventional methods.

EPrint Type:Conference or Workshop Item (Oral)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
ID Code:8836
Deposited By:Alexander Ilin
Deposited On:21 February 2012