Training humans to train robots dynamic motor skills

Student thesis: Master's ThesisMaster of Philosophy


Learning from Demonstration (LfD) is commonly considered to be a natural and intuitive way for novice users to teach motor skills to robots. However, it is important to acknowledge that the effectiveness of LfD is heavily dependent on the quality of demonstrations, something that may not be assured with novices. Ma-chine Teaching (MT) is one of the approaches to improve the quality of teaching by minimising both the teaching risk and the teaching effort. In MT, there is a human demonstrator, a machine teacher and a robot learner. Having knowledge of the target model and the learning algorithm, the machine teacher guides and/or trains the human demonstrator so that he or she can produce informative demonstrations. Although the use of MT has been shown to greatly improve teaching ability and learning efficiency in several applications, it remains an open question as to the most effective way of guiding demonstrators to produce informative demonstrations to teach dynamic control to robots. To this end, the aim of this project is to develop a MT system that can guide and train novice human demonstrators to improve their ability to teach motor skills to dynamically controlled robots through LfD.
The research question to be addressed in this project is:
Can Machine Teaching improve
Learning from Demonstration of robotic motor skills?
The hypothesis is that it is possible to develop a machine teacher that allows novice human demonstrators to teach dynamic control to robots through LfD with higher accuracy and lower effort than when following natural human teaching strategies.
To answer this research question, a MT problem of dynamic motor skill teaching with LfD using ridge regression is constructed by applying the MT formulation. Then, it is solved analytically for a case where the feature vector is 2-dimensional, and the demonstration optimality condition for higher dimensional cases is analysed. Finally, a MT system is developed to guide and train human teachers using an index to measure the quality of demonstrations proposed based on the solution to the MT problem.
The effectiveness of the proposed teaching framework was evaluated by applying it to the teaching of motor skills using a torque-controlled pendulum as a simple, dynamically controlled learner robot. The results of the simulated experiment showed that the error in the reproduced trajectory rapidly dropped as the teaching quality index derived using MT was maximised, suggesting that the in-dex accurately reflects the quality of the demonstrations. In an experiment with novice human demonstrators, guidance using the proposed teaching quality index significantly improved their selection of demonstrations and reduced the error in the skills acquired by the robot. Moreover, the guidance is shown to facilitate high-level learning for demonstrators to provide good demonstrations for teaching new motor skills.
This research project validates the effectiveness of the use of MT in guiding and training novice human teachers to teach simple dynamic motor control. In future work, this work may be extended to (i) teach more complex dynamic motor skills that require robots with more degrees of freedom or learning algorithms that do not have closed-form solutions, (ii) examine how different design choices for ma-chine teaching problems can be used to reflect the purpose and preferences of the user, and (iii) evaluate the effectiveness of different types of guiding and training methods using the MT framework.
Date of Award1 Oct 2021
Original languageEnglish
Awarding Institution
  • King's College London
SupervisorMatthew Howard (Supervisor) & Osvaldo Simeone (Supervisor)

Cite this