Model-Free Apprenticeship Learning for Transfer of Human Impedance Behaviour

TY - CHAP

T1 - Model-Free Apprenticeship Learning for Transfer of Human Impedance Behaviour

AU - Mori, Takeshi

AU - Howard, Matthew

AU - Vijayakumar, Sethu

PY - 2011

Y1 - 2011

N2 - We present a method for transferring behaviour from humans to robots via apprenticeship learning. While previous methods have relied on an accurate model of the demonstrator's dynamics, in most practical settings such models fail to capture (i) complex, non-linear dynamics of the hu- man musculoskeletal system, and (ii) inconsistencies between modelling assumptions and the configuration and placement of measurement apparatus. To avoid such issues, we propose a model-free approach to apprenticeship learning, in which off- policy, model-free reinforcement learning techniques are used to extract a model of the objective function optimised in human behaviour. As a key ingredient, we derive a novel formulation of Least Squares Policy Iteration (LSPI) and Least Squares Temporal Difference learning (LSTD) to enable their application in this setting. The robustness of our approach is demonstrated in experiments where human hitting behaviour is transferred to a non-biomorphic robotic device.

AB - We present a method for transferring behaviour from humans to robots via apprenticeship learning. While previous methods have relied on an accurate model of the demonstrator's dynamics, in most practical settings such models fail to capture (i) complex, non-linear dynamics of the hu- man musculoskeletal system, and (ii) inconsistencies between modelling assumptions and the configuration and placement of measurement apparatus. To avoid such issues, we propose a model-free approach to apprenticeship learning, in which off- policy, model-free reinforcement learning techniques are used to extract a model of the objective function optimised in human behaviour. As a key ingredient, we derive a novel formulation of Least Squares Policy Iteration (LSPI) and Least Squares Temporal Difference learning (LSTD) to enable their application in this setting. The robustness of our approach is demonstrated in experiments where human hitting behaviour is transferred to a non-biomorphic robotic device.

U2 - 10.1109/Humanoids.2011.6100830

DO - 10.1109/Humanoids.2011.6100830

M3 - Conference paper

SN - 978-1-61284-866-2

SP - 239

EP - 246

BT - 2011 11th IEEE-RAS International Conference on Humanoid Robots (Humanoids)

PB - IEEE

ER -