Using Dimensionality Reduction to Exploit Constraints in Reinforcement Learning

Sebastian Bitzer, Matthew Howard, Sethu Vijayakumar

Research output: Chapter in Book/Report/Conference proceedingConference paper

43 Citations (Scopus)


Reinforcement learning in the high-dimensional, continuous spaces typical in robotics, remains a challenging problem. To overcome this challenge, a popular approach has been to use demonstrations to find an appropriate initialisation of the policy in an attempt to reduce the number of iterations needed to find a solution. Here, we present an alternative way to incorporate prior knowledge from demonstrations of individual postures into learning, by extracting the inherent problem structure to find an efficient state representation. In particular, we use probabilistic, nonlinear dimensionality reduction to capture latent constraints present in the data. By learning policies in the learnt latent space, we are able to solve the planning problem in a reduced space that automatically satisfies task constraints. As shown in our experiments, this reduces the exploration needed and greatly accelerates the learning. We demonstrate our approach for learning a bimanual reaching task on the 19-DOF KHR-1HV humanoid.
Original languageEnglish
Title of host publication2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Place of PublicationPiscataway, N.J.
Number of pages7
ISBN (Print)9781424466740
Publication statusPublished - 2010


Dive into the research topics of 'Using Dimensionality Reduction to Exploit Constraints in Reinforcement Learning'. Together they form a unique fingerprint.

Cite this