Ensembles for sequence learning
PhD thesis, Ecole Polytechnique Federale de Lausanne.
This thesis explores the application of ensemble methods to sequential learning tasks. The focus is on
the development and the critical examination of new methods or novel applications of existing methods,
with emphasis on supervised and reinforcement learning problems.
In both types of problems, even after having observed a certain amount of data, we are often faced
with uncertainty as to which hypothesis is correct among all the possible ones. However, in many methods
for both supervised and for reinforcement learning problems this uncertainty is ignored, in the sense that
there is a single solution selected out of the whole of the hypothesis space. Apart from the classical solution
of analytical Bayesian formulations, ensemble methods offer an alternative approach to representing this
uncertainty. This is done simply through maintaining a set of alternative hypotheses.
The sequential supervised problem considered is that of automatic speech recognition using hidden
Markov models. The application of ensemble methods to the problem represents a challenge in itself, since
most such methods can not be readily adapted to sequential learning tasks. This thesis proposes a number
of different approaches for applying ensemble methods to speech recognition and develops methods for
effective training of phonetic mixtures with or without access to phonetic alignment data. Furthermore,
the notion of expected loss is introduced for integrating probabilistic models with the boosting approach.
In some cases substantial improvements over the baseline system are obtained.
In reinforcement learning problems the goal is to act in such a way as to maximise future reward in a
given environment. In such problems uncertainty becomes important since neither the environment nor
the distribution of rewards that result from each action are known. This thesis presents novel algorithms
for acting nearly optimally under uncertainty based on theoretical considerations. Some ensemble-based
representations of uncertainty (including a fully Bayesian model) are developed and tested on a few
simple tasks resulting in performance comparable with the state of the art. The thesis also draws some
parallels between a proposed representation of uncertainty based on gradient-estimates and on "prioritised
sweeping" and between the application of reinforcement learning to controlling an ensemble of classifiers
and classical supervised ensemble learning methods.