Statistical asymptotic and non-asymptotic consistency of bayesian networks : convergence to the right structure and consistent probability estimates
Sylvain Gelly and Olivier Teytaud
In: CAP 2005, Nice, France(2005).

## Abstract

The problem of calibrating relations from examples is a classical problem in learning theory. This problem has in particular been studied in the theory of empirical process (providing asymptotic results), and through statistical learning theory (\cite{KT},\cite{VC71}), and many more of less empirical methods. The application of learning theory to Bayesian networks is still uncompleted and we propose a contribution, especially through the use of covering numbers. We deduce multiples corollaries, among which : \begin{itemize} \item consistency of learning in Bayesian networks : which paradigms lead to consistency/universal consistency ? In particular, we propose an algorithm for which consistency is proved. Usual local calibrations are in fact not consistent. \item the choice of the structure of Bayesian network : how to ensure that the structure will converge asymptotically towards a not-too complex structure ? In particular, how to generate non-observable states simplifying the overall network ? In particular, we show the influence of a structural entropy on the covering numbers which is not taken into account in usual scores. \item the sample complexity in Bayesian networks : how many examples are required for a given precision on density estimation ? \item the convergence to the real dependency-structure : how to avoid that $P(B|A)$ is chosen as a parameter of the network whereas it is useless~? \end{itemize}