|
Interpretation of Hybrid Generative/Discriminative Algorithms
Jinghao Xue and Mike Titterington
Neurocomputing
2007.
AbstractIn discriminant analysis, probabilistic generative and
discriminative approaches represent two paradigms of statistical
modelling and learning. In order to exploit the best of both worlds,
hybrid modelling and learning techniques have attracted much
research interest recently, one example being the so-called hybrid
generative/discriminative algorithm proposed in~\citet{Raina:2003}
and its multi-class extension~\citep{Fujino:2007IPM}. In this paper,
we interpret this hybrid algorithm from three perspectives, namely
class-conditional probabilities, class-posterior probabilities and
loss functions underlying the model. We suggest that the hybrid
algorithm is by nature a generative model with its parameters learnt
through both generative and discriminative approaches, in the sense
that in fact it assumes a scaled data-generation process and uses
scaled class-posterior probabilities to perform discrimination. Our
suggestion can also be applied to the multi-class extension. In
addition, using simulated data, we compare the performance of the
normalised hybrid algorithm as a classifier with those of the
na\"{i}ve Bayes classifier and linear logistic regression. In
general, our simulation studies suggest the following: if the
covariance matrices are diagonal matrices, the na\"{i}ve Bayes
classifier performs the best; if the covariance matrices are
non-diagonal matrices, linear logistic regression performs the best.
In other words, our studies cannot support that the hybrid algorithm
offers better performance than either the na\"{i}ve Bayes classifier
or linear logistic regression alone, a phenomenon observed from the
empirical studies in~\citet{Raina:2003} [Edit]
|
