PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

On the properties of the Bethe approximation and loopy belief propagation on binary networks
Joris Mooij and Bert Kappen
Journal of Statistical Mechanics: Theory and Experiment Volume 2005, Number 11, P11012, 2005. ISSN 1742-5468


We analyse the local stability of the high-temperature fixed point of the loopy belief propagation (LBP) algorithm and how this relates to the properties of the Bethe free energy which LBP tries to minimize. We focus on the case of binary networks with pairwise interactions. In particular, we state sufficient conditions for convergence of LBP to a unique fixed point and show that these are sharp for purely ferromagnetic interactions. In contrast, in the purely antiferromagnetic case, the undamped parallel LBP algorithm is suboptimal in the sense that the stability of the fixed point breaks down much earlier than for damped or sequential LBP; we observe that the onset of instability for the latter algorithms is related to the properties of the Bethe free energy. For spin-glass interactions, damping LBP only helps slightly. We estimate analytically the temperature at which the high-temperature LBP fixed point becomes unstable for random graphs with arbitrary degree distributions and random interactions.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
Postscript - Requires a viewer, such as GhostView
EPrint Type:Article
Additional Information:J. Stat. Mech. © copyright (2005) IOP Publishing Ltd.
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:1936
Deposited By:Joris Mooij
Deposited On:30 December 2005