PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

WIKImage : correlated image and text datasets
Doni Pracner, Nenad Tomašev, Miloš Radovanović, Dunja Mladenić and Mirjana Ivanović
In: IS 2011, 8-12 Oct 2011, Ljubljana, Slovenia.

Abstract

This paper presents work towards the creation of free and redistributable datasets of correlated images and text. Collections of free images and related text were extracted from Wikipedia with our new tool WIKImage. An additional tool – WIKImage browser – was introduced to visualize the resulting dataset, and was expanded into a manual labeling tool. The paper presents a starting dataset of 1007 images labeled with any combination of 14 tags. The images were processed into a number of scale invariant (SIFT) and color histogram features, and the captions were transformed into a bag-of-words (BOW) representation. Experiments were then performed with the aim of classifying data with respect to each of the labels on dataset variants with just the image information, just the textual data, and both, in order to estimate the difficulty of the dataset in the context of different feature spaces. Results indicate improvements in precision, recall and the F-measure when using the combined representation with support vector machines as well as the k-nearest neighbor classifier with the cosine similarity measure.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Information Retrieval & Textual Information Access
ID Code:8724
Deposited By:Jan Rupnik
Deposited On:21 February 2012