Phase Transitions within Grammatical Inference
N Pernot, Antoine Cornuéjols and Michele Sebag
In: IJCAI 2005, 01-05 August 2005, Edimburg, UK..
It is now well-known that the feasibility of inductive
learning is ruled by statistical properties linking
the empirical risk minimization principle and
the “capacity” of the hypothesis space. The discovery,
a few years ago, of a phase transition phenomenon
in inductive logic programming proves
that other fundamental characteristics of the learning
problems may similarly affect the very possibility
of learning under very general conditions.
Our work examines the case of grammatical inference.
We show that while there is no phase
transition when considering the whole hypothesis
space, there is a much more severe “gap” phenomenon
affecting the effective search space of
standard grammatical induction algorithms for deterministic
finite automata (DFA). Focusing on the
search heuristics of the RPNI and RED-BLUE algorithms,
we show that they overcome this problem
to some extent, but that they are subject to overgeneralization.
The paper last suggests some directions
for new generalization operators, suited to this
Phase Transition phenomenon.