PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Can relevance of images be inferred from eye movements?
Arto Klami, Craig Saunders, Teofilo de Campos and Samuel Kaski
In: ACM International Conference on Multimedia Information Retrieval, 30-31 Oct 2008, Vancouver, Canada.


Query formulation and efficient navigation through data to reach relevant results are undoubtedly major challenges for image or video retrieval. Queries of good quality are typically not available and the search process needs to rely on relevance feedback given by the user, which makes the search process iterative. Giving explicit relevance feedback is laborious, not always easy, and may even be impossible in ubiquitous computing scenarios. A central question then is: Is it possible to replace or complement scarce explicit feedback with implicit feedback inferred from various sensors not specifically designed for the task? In this paper, we present preliminary results on inferring the relevance of images based on implicit feedback about users' attention, measured using an eye tracking device. It is shown that, in reasonably controlled setups at least, already fairly simple features and classifiers are capable of detecting the relevance based on eye movements alone, without using any explicit feedback.

EPrint Type:Conference or Workshop Item (Poster)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:User Modelling for Computer Human Interaction
Multimodal Integration
Information Retrieval & Textual Information Access
ID Code:4612
Deposited By:Arto Klami
Deposited On:13 March 2009