PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Modeling Individual and Group Action in Meetings: a Two-layer HMM Framework
Dong Zhang, Daniel Gatica-Perez, Samy Bengio, Iain McCowan and Guillaume Lathoud
In: IEEE Workshop on Event Mining: Detection and Recognition of Events in Video (In Association with CVPR 2004), June 2004, Washington, USA.

There is a more recent version of this eprint available. Click here to view it.


We address the problem of recognizing sequences of human interaction patterns in meetings, with the goal of structuring them in semantic terms. The investigated patterns are inherently group-based (defined by the individual activities of meeting participants, and their interplay), and multimodal (as captured by cameras and microphones). By defining a proper set of individual actions, group actions can be modeled as a two-layer process, one that models basic individual activities from low-level audio-visual features, and another one that models the interactions. We propose a two-layer Hidden Markov Model (HMM) framework that implements such concept in a principled manner, and that between has advantages over previous works. First, by decomposing the problem hierarchically, learning is performed on low-dimensional observation spaces, which results in simpler models. Second, our framework is easier to interpret, as both individual and group actions have a clear meaning, and thus easier to improve. Third, different HMM models can be used in each layer, to better reflect the nature of each subproblem. Our framework is general and extensible, and we illustrate it with a set of eight group actions, using a public five-hour meeting corpus. Experiments and comparison with a single-layer HMM baseline system show its validity.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Talk)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
Learning/Statistics & Optimisation
Multimodal Integration
ID Code:124
Deposited By:Samy Bengio
Deposited On:27 May 2004

Available Versions of this Item