Large-Margin Structured Prediction via Linear Programming
Zhuoran Wang and John Shawe-Taylor
In: Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS 2009), 16-18 Apr 2009, Clearwater Beach, Florida, USA.
This paper presents a novel learning algorithm for structured
classification, where the task is to predict multiple and
interacting labels (multilabel) for an input object. The problem
of finding a large-margin separation between correct multilabels
and incorrect ones is formulated as a linear program. Instead of
explicitly writing out the entire problem with an exponentially
large constraint set, the linear program is solved iteratively via
column generation. In this case, the process of generating most
violated constraints is equivalent to searching for highest-scored
misclassified incorrect multilabels, which can be easily achieved
by decoding the structure based on current estimations. In
addition, we also explore the integration of column generation and
an extragradient method for linear programming to gain further
efficiency. The proposed method has the advantages that it can
handle arbitrary structures and larger-scale problems.
Experimental results on part-of-speech tagging and statistical
machine translation tasks are reported, demonstrating the
competitiveness of our approach.