PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Subtitle-free Movie to Script Alignment
Pramod Sankar K, C. V. Jawahar and Andrew Zisserman
In: British Machine Vision Conference (BMVC), 07-10 Sep 2009, London, UK.


A standard solution for aligning scripts to movies is to use dynamic time warping with the subtitles (Everingham et al., BMVC 2006). We investigate the problem of aligning scripts to TV video/movies in cases where subtitles are not available, e.g. in the case of silent films or for film passages which are non-verbal. To this end we identify a number of "modes of alignment" and train classifiers for each of these. The modes include visual features, such as locations and face recognition, and audio features such as speech. In each case the feature gives some alignment information, but is too noisy when used independently. We show that combining the different features into a single cost function and optimizing this using dynamic programming, leads to a performance superior to each of the individual features. The method is assessed on episodes from the situation comedy Seinfeld, and on Charlie Chaplin and Indian movies.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Oral)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
Learning/Statistics & Optimisation
ID Code:6560
Deposited By:Karteek Alahari
Deposited On:08 March 2010