Improved Learning of Gaussian-Bernoulli Restricted Boltzmann Machines
KyungHyun Cho, Alexander Ilin and Tapani Raiko
In: International Conference on Artificial Neural Networks (ICANN 2011), 14-17 Jun 2011, Espoo, Finland.

Abstract

We propose a few remedies to improve training of Gaussian-Bernoulli restricted Boltzmann machines (GBRBM), which is known to be difficult. Firstly, we use a different parameterization of the energy function, which allows for more intuitive interpretation of the parameters and facilitates learning. Secondly, we propose parallel tempering learning for GBRBM. Lastly, we use an adaptive learning rate which is selected automatically in order to stabilize training. Our extensive experiments show that the proposed improvements indeed remove most of the difficulties encountered when training GBRBMs using conventional methods.

[Edit]