PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Spelling It Out: Real-Time ASL Fingerspelling Recognition
Nicolas Pugeault and Richard Bowden
In: 1st IEEE Workshop on Consumer Depth Cameras for Computer Vision, jointly with ICCV'2011, 12 Nov 2011, Barcelona, Spain.


This article presents an interactive hand shape recognition user interface for American Sign Language (ASL) finger-spelling. The system makes use of a Microsoft Kinect device to collect appearance and depth images, and of the OpenNI+NITE framework for hand detection and tracking. Hand-shapes corresponding to letters of the alphabet are characterized using appearance and depth images and classified using random forests. We compare classification using appearance and depth images, and show a combination of both lead to best results, and validate on a dataset of four different users. This hand shape detection works in real-time and is integrated in an interactive user interface allowing the signer to select between ambiguous detections and integrated with an English dictionary for efficient writing.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Poster)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
Learning/Statistics & Optimisation
ID Code:9036
Deposited By:Nicolas Pugeault
Deposited On:21 February 2012