PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Structure inference for Bayesian multisensory scene understanding
Timothy Hospedales and Sethu Vijayakumar
IEEE Transactions on Pattern Analysis and Machine Intelligence 2007.

Abstract

We investigate a solution to the problem of multi-sensor perception and tracking by formulating it in the framework of Bayesian model selection. Humans robustly associate multi-sensory data as appropriate, but previous theoretical work has focused largely on purely integrative cases, leaving segregation unaccounted for and unexploited by machine perception systems. We illustrate a unifying, Bayesian solution to multisensor perception and tracking which accounts for both integration and segregation by explicit probabilistic reasoning about data association in a temporal context. Explicit inference of multisensory data association may also be of intrinsic interest for higher level understanding of multisensory data. We illustrate this using a probabilistic model of audio-visual data in which unsupervised learning and inference provide automatic audio-visual detection and tracking of two human subjects, speech segmentation, and association of each conversational segment with the speaking person.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
Multimodal Integration
ID Code:3434
Deposited By:Timothy Hospedales
Deposited On:10 February 2008