PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

On Discriminative Bayesian Network Classifiers and Logistic Regression
Teemu Roos, Hannes Wettig, Peter Grünwald, Petri Myllymäki and Henry Tirri
On Discriminative Bayesian Network Classifiers and Logistic Regression Number 59, pp. 267-296, 2005.


Discriminative learning of the parameters in the Naive Bayes model is known to be equivalent to a logistic regression problem. Here we show that the same fact holds for much more general Bayesian network models, as long as the corresponding network structure satisfies a certain graph-theoretic property. The property holds for Naive Bayes but also for more complex structures such as tree-augmented Naive Bayes (TAN) as well as for mixed diagnostic-discriminative structures. Our results imply that for networks satisfying our property, the conditional likelihood cannot have local maxima so that the global maximum can be found by simple local optimization methods. We also show that if this property does NOT hold, then in general the conditional likelihood CAN have local, non-global maxima. We illustrate our theoretical results by empirical experiments with local optimization in a conditional Naive Bayes model. Furthermore, we provide a heuristic strategy for pruning the number of parameters and relevant features in such models. For many data sets, we obtain good results with heavily pruned submodels containing much less parameters than the original naive Bayes model.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:122
Deposited By:Hannes Wettig
Deposited On:27 May 2004