PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Bayesian Networks and Inner Product Spaces
Atsuyoshi Nakamura, Michael Schmitt, Niels Schmitt and Hans Simon
Journal of Machine Learning Research Volume 6, pp. 1383-1403, 2005.


Bayesian networks have become one of the major models for statistical inference. We study the question whether the decisions computed by a Bayesian network can be represented within a low-dimensional inner product space. We focus on two-label classification tasks over the Boolean domain. As the main results, we establish upper and lower bounds on the dimension of the ``natural'' inner product space for Bayesian networks with an explicitly given (full or reduced) parameter collection. In particular, these bounds are tight up to a factor of 2. For some nontrivial cases, we even determine the exact values of this dimension. Further, we consider a variant of the logistic autoregressive Bayesian network and show that every sufficiently expressive inner product space must have dimension at least $2^{\Omega(n)}$, where n is the number of network nodes. As a major technical contribution, this work reveals combinatorial and algebraic structures within Bayesian networks such that known methods for the derivation of lower bounds on the dimension of inner product spaces can be brought into play.

EPrint Type:Article
Additional Information:A preliminary (and less detailed) version of this paper was published in the Proceedings of COLT 2004.
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:2216
Deposited By:Hans Simon
Deposited On:29 September 2006