3D Tracking of Human Hands in Interaction with Unknown Objects

Brief description

The analysis and the understanding of object manipulation scenarios based on computer vision techniques can be greatly facilitated if we can gain access to the full articulation of the manipulating hands and the 3D pose of the manipulated objects. Currently, there exist methods for tracking hands in interaction with objects whose 3D models are known. There are also methods that can reconstruct 3D models of objects that are partially observable in each frame of a sequence. However, to the best of our knowledge, no method can track hands in interaction with unknown objects. In this paper we propose such a method. Experimental results show that hand tracking can be achieved with an accuracy that is comparable to the one obtained by methods that assume knowledge of the object models. Additionally, as a by-product, the proposed method delivers accurate 3D models of the manipulated objects.

Sample results

Video with experimental results

Contributors

• Paschalis Panteleris, Nikolaos Kyriazis, Antonis A. Argyros
• This work has been supported by the EU project ROBOHOW.

Relevant publications

• P. Panteleris, N. Kyriazis, A.A. Argyros, “3D Tracking of Human Hands in Interaction with Unknown Objects”, British Machine Vision Conference (BMVC 2015), Swansea, UK, Sep. 7-10, 2015.
• P. Panteleris, N. Kyriazis, A.A. Argyros, "Recovering 3D models of manipulated objects through 3D tracking of hand-object interaction", In IEEE International Conference on Computer Vision Workshops (OUI 2015 - ICCVW 2015), IEEE, Santiago, Chile, November 2015.

The electronic versions of the above publications can be downloaded from my publications page.