PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Can relevance of images be inferred from eye movements?
Arto Klami, Craig Saunders, Teofilo E. de Campos and Samuel Kaski
ACM International Conference on Multimedia Information Retrieval 2008.


Query formulation and efficient navigation through data to reach relevant results are undoubtedly major challenges for image or video retrieval. Queries of good quality are typically not available and the search process needs to rely on relevance feedback given by the user, which makes the search process iterative. Giving explicit relevance feedback is laborious, not always easy, and may even be impossible in ubiquitous computing scenarios. A central question then is: Is it possible to replace or complement scarce explicit feedback with implicit feedback inferred from various sensors not specifically designed for the task? In this paper, we present preliminary results on inferring the relevance of images based on implicit feedback about users' attention, measured using an eye tracking device. It is shown that, in reasonably controlled setups at least, already fairly simple features and classifiers are capable of detecting the relevance based on eye movements alone, without using any explicit feedback.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
User Modelling for Computer Human Interaction
Multimodal Integration
ID Code:5329
Deposited By:Teofilo de Campos
Deposited On:31 March 2009