Monday, November 10, 2014 - 14:30

Machine learning for social multi-user human robot interaction

Abstract: With the introduction of affordable humanoid robots into everyday life, the need for not only effective, but also socially intelligent behaviour in such robots is becoming more and more apparent.  In the EC FP7 project JAMES (Joint Action for Multimodal Embodied Systems) we have developed a robot bartender system with the aim of studying social, multi-party, human-robot interaction.  In particular, we have developed data-driven techniques for social state monitoring and social skills execution, the central components of the system.  For social state monitoring for example, we have built classifiers for deciding if a customer is seeking attention from the bartender, based on the available audio-visual input.  I will focus however on the social skills execution part and present a hierarchical form of reinforcement learning of action selection strategies for handling the orders of multiple customers in a socially appropriate manner.  This work also involved the development of a multi-user simulated environment for training and evaluating such strategies.  Finally, I will discuss recent experiments aimed at handling uncertain input in multi-user human-robot interaction and outline how this work can be taken further.


Simon Keizer is a Research Fellow in the Interaction Lab in the School of Mathematical and Computer Sciences (MACS) at Heriot-Watt University, Edinburgh (UK). He has an MSc in Applied Mathematics and a PhD in Computer Science, both obtained at the University of Twente (NL). The main focus of his research is on interaction management and user simulation for training and evaluating spoken dialogue systems and in particular the use of machine learning techniques in this area. As a Research Associate at Tilburg University (NL) and Cambridge University (UK), he has been involved in both national and international collaborative research projects. In the past three-and-a-half years, he worked on the EU FP7 project JAMES, focusing on machine learning techniques for multi-user social human robot interaction.