PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Taking the bite out of automatic naming of characters in TV video
Mark Everingham, Josef Sivic and Andrew Zisserman
Image and Vision Computing Volume 27, Number 5, pp. 545-559, 2009.


We investigate the problem of automatically labelling appearances of characters in TV or film material with their names. This is tremendously challenging due to the huge variation in imaged appearance of each character and the weakness and ambiguity of available annotation. However, we demonstrate that high precision can be achieved by combining multiple sources of information, both visual and textual. The principal novelties that we introduce are: (i) automatic generation of time stamped character annotation by aligning subtitles and transcripts; (ii) strengthening the supervisory information by identifying when characters are speaking. In addition, we incorporate complementary cues of face matching and clothing matching to propose common annotations for face tracks, and consider choices of classifier which can potentially correct errors made in the automatic extraction of training data from the weak textual annotation. Results are presented on episodes of the TV series “Buffy the Vampire Slayer”.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
ID Code:5402
Deposited By:Karteek Alahari
Deposited On:11 June 2009