Scene Segmentation with CRFs Learned from Partially Labeled Images
Jakob Verbeek and William Triggs
In: Neural Information Processing Systems (NIPS), 3-10 Dec 2007, Vancouver, Canada.
Conditional Random Fields (CRFs) are an effective tool for a variety
of different data segmentation and labeling tasks including visual
scene interpretation, which seeks to partition images into their
constituent semantic-level regions and assign appropriate class labels
to each region. For accurate labeling it is important to capture the
global context of the image as well as local information. We introduce
a CRF based scene labeling model that incorporates both local
features and features aggregated over the whole image or large
sections of it. Secondly, traditional CRF learning requires fully
labeled datasets which can be costly and troublesome to produce. We
introduce a method for learning CRFs from datasets with many unlabeled
nodes by marginalizing out the unknown labels so that the
log-likelihood of the known ones can be maximized by gradient ascent.
Loopy Belief Propagation is used to approximate the marginals needed
for the gradient and log-likelihood calculations and the Bethe
free-energy approximation to the log-likelihood is monitored to
control the step size. Our experimental results show that effective
models can be learned from fragmentary labelings and that
incorporating top-down aggregate features significantly improves the
segmentations. The resulting segmentations are compared to the
state-of-the-art on three different image data sets.