Brief description

We have developed a method for detecting the fingers in human hands. Hands are detected based on skin-colored regions detection and tracking. Fingers are then detected based on multi-scale processing of blob-contours. Combined with the 3D tracking capabilities of the skin-color detector and tracker, finger detection can give very useful information on the 3D position of fingertips. This type of information is particularly useful in the context of Cognitive vision systems such as the ActIPret demonstrator, whose goal is the interpretation of the activities of people handling tools. Moreover, finger detection can provide rich perceptual input to gesture recognition systems. We have already developed a system that permits to a human to control the mouse of a computer. The developed demonstrator has successfully been employed in real-world situations where a human controls the computer during MS powerpoint presentations.

A preliminary version of the proposed tracker has been presented in the ECCV’04 demonstrations session. This has also been used to develop a prototype vision system that controls a computer mouse based on interpretation of hand gestures.

Sample results

Finger detection on the tracked skin-colored regions.

Contributors

Antonis Argyros, Manolis Lourakis. Many thanks to the ActIPret consortium for the fruitful interaction.

Relevant publications

  • A.A. Argyros, M.I.A. Lourakis, “Vision-based Interpretation of Hand Gestures for Remote Control of a Computer Mouse”, in proceedings of the HCI’06 workshop (in conjunction with ECCV’06), LNCS 3979, Springer Verlag, pp.40-51, Graz, Austria, May 13th, 2006. Recipient of the “Best Paper Award”.
  • A.A. Argyros, M.I.A. Lourakis, “Tracking Skin-colored Objects in Real-time”, invited contribution to the “Cutting Edge Robotics” book, ISBN 3-86611-038-3, Advanced Robotic Systems International, 2005.
  • A.A. Argyros, M.I.A. Lourakis, “Real time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera”, in proceedings of the European Conference on Computer Vision (ECCV’04), Springer-Verlag, vol. 3, pp. 368-379, May 11-14, 2004, Prague, Chech Republic.
  • A.A. Argyros, M.I.A. Lourakis, “Tracking Multiple Colored Blobs With a Moving Camera” in proceedings of the Computer Vision and Pattern Recognition Conference, (CVPR’05), vol. 2, no. 2, p. 1178, San Diego, USA, June 20-26, 2005.
  • A.A. Argyros, M.I.A. Lourakis, “3D Tracking of Skin-Colored Regions by a Moving Stereoscopic Observer”, Applied Optics, Information Processing Journal, Special Issue on Target Detection, Vol. 43, No 2, pp. 366-378, January 2004.
  • K. Sage, J. Howell, H. Buxton, A.A. Argyros, “Learning Temporal Structure for Task-based Control”, Image and Vision Computing Journal (IVC), special issue on Cognitive Systems, conditionally accepted, under revision.
  • S.O. Orphanoudakis, A.A. Argyros, M. Vincze “Towards a Cognitive Vision Methodology: Understanding and Interpreting Activities of Experts”, ERCIM News, No 53, Special Issue on “Cognitive Systems, April 2003

The electronic versions of the above publications can be downloaded from my publications page.