PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Multi-modal visual concept classification of images via Markov random walk over tags
Motoaki Kawanabe, Alexander Binder, Christina Mueller and Wojciech Wojcikiewicz
Applications of Computer Vision (WACV), 2011 IEEE Workshop on 2011. ISSN 1550-5790


Automatic annotation of images is a challenging task in computer vision because of “semantic gap” between highlevel visual concepts and image appearances. Therefore, user tags attached to images can provide further information to bridge the gap, even though they are partially uninformative and misleading. In this work, we investigate multi-modal visual concept classification based on visual features and user tags via kernel-based classifiers. An issue here is how to construct kernels between sets of tags. We deploy Markov random walks on graphs of key tags to incorporate co-occurrence between them. This procedure acts as a smoothing of tag based features. Our experimental result on the ImageCLEF2010 PhotoAnnotation benchmark shows that our proposed method outperforms the baseline relying solely on visual information and a recently published state-of-the-art approach.

EPrint Type:Article
Additional Information:Link:
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
Multimodal Integration
Information Retrieval & Textual Information Access
ID Code:8010
Deposited By:Alexander Binder
Deposited On:17 March 2011