Robust and efficient transfer learning with hidden parameter Markov decision processes

Conference Paper

Copyright © 2017, Association for the Advancement of Artificial Intelligence ( All rights reserved. An intriguing application of transfer learning emerges when tasks arise with similar, but not identical, dynamics. Hidden Parameter Markov Decision Processes (HiP-MDP) embed these tasks into a low-dimensional space; given the embedding parameters one can identify the MDP for a particular task. However, the original formulation of HiP-MDP had a critical flaw: the embedding uncertainty was modeled independently of the agent's state uncertainty, requiring an arduous training procedure. In this work, we apply a Gaussian Process latent variable model to jointly model the dynamics and the embedding, leading to a more elegant formulation, one that allows for better uncertainty quantification and thus more robust transfer.

Duke Authors

Cited Authors

  • Killian, TW; Konidaris, G; Doshi-Velez, F

Published Date

  • January 1, 2017

Published In

  • 31st AAAI Conference on Artificial Intelligence, AAAI 2017

Start / End Page

  • 4949 - 4950

Citation Source

  • Scopus