PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Retrieval of Multimedia Objects by Combining Semantic Information from Visual and Textual Descriptors
Mats Sjöberg, Jorma Laaksonen, Matti Pöllä and Timo Honkela
In: 16th International Conference on Artificial Neural Networks (ICANN 2006), 10 Sep - 14 Sep 2006, Athens, Greece.


We propose a method of content-based multimedia retrieval of objects with visual, aural and textual properties. In our method, training examples of objects belonging to a specific semantic class are associated with their low-level visual descriptors (such as MPEG-7) and textual features such as frequencies of significant keywords. A fuzzy mapping of a semantic class in the training set to a class of similar objects in the test set is created by using Self-Organizing Maps (SOMs) trained from automatically extracted low-level descriptors. We have performed several experiments with different textual features to evaluate the potential of our approach in bridging the gap from visual features to semantic concepts by the use textual presentations. Our initial results show a promising increase in retrieval performance.

EPrint Type:Conference or Workshop Item (Oral)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
Natural Language Processing
Information Retrieval & Textual Information Access
ID Code:2588
Deposited By:Jorma Laaksonen
Deposited On:14 February 2008