Research
in Human-Robot Interaction has mainly addressed visual competences
involved in interaction scenarios. Most recent work has been pursued
within the INDIGO research project, which I coordinated.
INDIGO
was an EC funded research project that finished in January 2010. The
goal of INDIGO was to develop technology to facilitate the advancement
of human-robot interaction. This was achieved both by enabling robots to
perceive natural human behavior as well as by making them act in ways
that are familiar to humans.
Tracking of
the upper human body is one of the most interesting and challenging
research fields in computer vision and comprises an important component
used in gesture recognition applications. In this work, a probabilistic
approach towards arm and hand tracking is prefsented. We propose the use
of a kinematics model together with a segmentation of the parameter
space to cope with the space dimensionality problem. Moreover, the
combination of particle filters with hidden Markov models enables the
simultaneous tracking of several hypotheses for the body orientation and
the configuration of each of the arms.
Experiment 1
Experiment 2
Experiment 3
Hand/Face tracking through propagation of pixel hypotheses
The
proposed approach difers significantly from existing ones on important
aspects of the representation of the location and the shape of tracked
objects and of the uncertainty associated with them. The location and
the speed of each object is modeled as a discrete time, linear dynamical
system which is tracked using Kalman filtering. Information about the
spatial distribution of the pixels
of each tracked object is passed on from frame to frame by propagating a
set of pixel hypotheses, uniformly sampled from the original object's
projection to the target frame using the object's current dynamics, as
estimated by the Kalman filter. The density of the propagated pixel
hypotheses provides a novel metric that is used to associate image
pixels with existing object tracks by taking into account both the shape
of each object and the uncertainty associated with its track. The
proposed tracking approach has been developed to support face and hand
tracking for human-robot interaction. Nevertheless, it is readily
applicable to a much broader class of multiple objects tracking
problems.
Distinguishing between hands and faces - Hand Gesture Recognition
Our hand and face tracker (described above) provides a set of blob
hypotheses that correspond to the location of hands and faces of people
that are in front of the robot. To proceed with higher-level tasks like
hand gesture recognition, one has to distinguish between hypotheses that
belong to hands and hypotheses that belong to faces. Moreover, for hand
hypotheses, one has to know which hypotheses belong to left hands and
which hypotheses belong to right hands.
Towards this goal, we have developed a technique that incrementally
classifes a hypothesis into one of three classes: faces, left hands and
right hands. The incremental classifier provides a way to compute a
belief about the class of each hypothesis based on a set of featuresthat
contain information about the hypothesis shape, location and speed.
For each new frame, the belief is incrementaly updated based on the
belief of the previous frame and the current observations.
M. Sigalas, H. Baltzakis, and P. Trahanias, Visual tracking of independently moving body and arms, IIn Proc. IEEE/RSJ Intlernational Conference on Intelligent Robotics and Systems (IROS), St. Louis, MO, USA, October 2009.
Using multi-hypothesis mapping to close loops in complex cyclic environments
The
method consists of two phases. During the first phase, the algorithm
creates and tracks a number of possible robot paths along with their
corresponding maps. After all data is processed, the algorithm decides
which of the robot paths is most probable. During the second phase of
the method, an EM procedure is used in order to rectify the robot's path
and the corresponding map.
Mobile robot localization using Switching State-Space Modeling
In
order to carry out complex navigational tasks, an autonomous robotic
agent must be able to provide answers to the "Where am I?" question,
that is, to localize itself within its environment.
To reduce the inherent complexity associated with this problem, adoption
of appropriate geometric constrains in combination with effective
modelling of related information is required. The achieved abstraction,
not only makes robotic problems computationally feasible, but also
provides robustness in the presence of noise, or other, often
unpredictable, factors. Two of the most successful probabilistic models
proposed for this purpose in the past are generally fall into two major
categories: Hidden Markov Models (HMM) and Kalman filters. Kalman filter
approaches are bettern with respect to computational efficiency,
scalability, and accuracy. On the other hand, HMM-based approaches were
proved to be more robust in the presence of noise and/or unreliable
odometry information.
To combine the advantages from both of the above-mentioned approaches,
we have proposed a probabilistic framework for modelling the robot's
state and sensory information, based on Switching State-Space Models. A
central concept in our framework is to let HMM models handle the
qualitative aspects of the problem, i.e. perform coarse localization,
and Kalman filters the metric aspects, that is, elaborate on the
previous result and provide accurate localization. Discrete HMM update
equations are used in order to update the probabilities assigned to a
fixed, small number of discrete states, while Kalman filter based
trackers, operating within each discrete state, are used in order to
provide accurate metric information.
High-level features, consisting of sequences of line segments and corner
points, extracted robustly from laser range data, are used to
facilitate the implementation of the model, while, a fast dynamic
programming algorithm is used in order to produce and score matches
between them and an a-priori map. Experimental results have shown the
applicability of the algorithm for indoor navigation tasks where the
global localization capabilities of the HMM approaches and efficiency
and accuracy of the Kalman filter based approaches are required at the
same time.
Brain
mechanisms that facilitate acquiring a motor repertory by observational
learning are investigated and implementations of the computational
equivalents of these mechanisms in robotic agents are pursued. Our
modeling work focuses on neuroscientific experiments accomplishing the
computational reproduction of behavioral and brain-imaging data. In
particular, we have implemented a brain-like model that reproduces
bilateral activations across parieto-frontal-prefrontal pathways and
accomplishes:
executing a range of different behaviors,
recognizing the same behaviors when executed by others,
using bilateral activity to inhibit self-action during observation.
Recent research
work addresses the capacity of artificial systems to sense time and
processes the temporal characteristics of real world phenomena. Time
perception is a fundamental aspect of natural cognition that has been
rarely considered in the context of artificial systems. Using multiple
experimental setups we have investigated how duration perception
capacities modulate the acquisition, representation and abstraction of
behavioral skills, in self-organized neural network structures
developing brain-like properties. The obtained results are related with
brain imaging studies, and they are used to formulate suggestions for
the cortical representation of time in biological agents.
M. Maniadakis, M. Wittmann, P. Trahanias, "Time Experiencing by Robotic Agents", in Proc. European Symposium on Artificial Neural Networks (ESANN), 2011.
We have
investigated self-organized
mechanisms accomplishing self-monitoring and strategy control in a
multiple behavioral-rule switching paradigm (rules SS, OS, NR). Two
different schemes for rule representation and switching mechanisms have
appeared in a series of independently self-organized models. The
obtained results provide significant feedback to neuroscience suggesting
that when two people understand a given problem in different ways, then
their brains work differently when solving the underlying problem (this
is particularly relevant for high level cognitive skills).
Robot MUFIK at the Natural History Museum of Crete
Since
18-Jan-2010, MUFIK, an autonomously navigating robot, has been
installed at the premises of the Natural History Museum of Crete at
Heraklion. This robot runs as a permanent installation at the museum,
interacting with visitors via the touch screen and a simple simulated
face, autonomously guiding them at the foyer of the museum and showing
them around.
This robot runs completely unattended and it is operated on a daily
basis by the personnel of the museum. In the next few months (i.e.
within the spring of 2010) another, identical robot will be installed at
different section of the same museum, intended to offer guided tours to
the exhibits of this particular section.
INDIGO project: Evaluation sessions at FHW
The INDIGO
project addressed the development of Human-robot interaction
technology. It involved multi-modal, bi-directional interaction.
Moreover, it employed a mechanical head capable of mimicking human
facial expressions, and supporting naturalistic spoken conversation. The
head was embodied on a mobile robot empowered with advanced autonomous
navigation skills. The overall system was able to act according to
motion patterns that are familiar to humans.
Advanced natural dialogue capabilities facilitated the overall goal of
human-robot interaction. Natural dialogue involved input and output from
various modalities, such as spoken natural language, gestures,
emotions, and facial expressions. While the emphasis was on technologies
that allow robots to generate natural descriptions of their physical
surroundings, INDIGO also addressed interpretation of a relatively broad
range of input.
Emphasis was given in the creation of appropriate user models for humans
interacting with a robot as well as for the robot itself. User models
were used to drive the dialogue management system and, thus, to allow
adaptation in the behavior of the robot according to the perceived user
profile as well as the knowledge, personality and gathered experience of
the robot itself.
INDIGO was demonstrated by deploying a prototype system at the Hellenic Cosmos,
a Cultural Centre located in Athens. The prototype operated
autonomously, interacting with humans inexperienced in robots, offering
them the possibility to engage with advanced robotics technologies.
Three extensive evaluation sessions took place in the period
June-December 2009.
73rd International fair of Thessaloniki
From
September 5th to September 14th 2008, a prototype system was exhibited
at the international fair of Thessaloniki, which is the most prestigious
trade fair held in Greece . During the event, various software modules
including autonomous navigation, people tracking, and vision modules
were demonstrated. A simplified dialogue management system was also
installed offering human-robot-interaction capabilities to the visitors
of the exhibition. Input from the users was mostly given through the
touch screen interface. The exhibited prototype was able to operate for
more than 10 hours each day without any significant problems, being a
constant attractor of visitors - mostly children - that wanted to "play"
with it.
P.E. Trahanias, W. Burgard, D. Haehnel, M. Moors, D. Schulz, H. Baltzakis and A. Argyros, Interactive Tele-Presence in Exhibitions through Web-operated Robots,
in proceedings of the 11th International Conference on Advanced
Robotics (ICAR03), invited session on Robotics and Art, pp. 1253-1258,
University of Coimbra, Coimbra, Portugal, June 30 - July 3, 2003.
