Skip to main content

Robustly Learning Composable Options in Deep Reinforcement Learning

Publication ,  Conference
Bagaria, A; Senthil, J; Slivinski, M; Konidaris, G
Published in: IJCAI International Joint Conference on Artificial Intelligence
January 1, 2021

Hierarchical reinforcement learning (HRL) is only effective for long-horizon problems when high-level skills can be reliably sequentially executed. Unfortunately, learning reliably composable skills is difficult, because all the components of every skill are constantly changing during learning. We propose three methods for improving the composability of learned skills: representing skill initiation regions using a combination of pessimistic and optimistic classifiers; learning re-targetable policies that are robust to non-stationary subgoal regions; and learning robust option policies using model-based RL. We test these improvements on four sparse-reward maze navigation tasks involving a simulated quadrupedal robot. Each method successively improves the robustness of a baseline skill discovery method, substantially outperforming state-of-the-art flat and hierarchical methods.

Duke Scholars

Published In

IJCAI International Joint Conference on Artificial Intelligence

ISSN

1045-0823

ISBN

9780999241196

Publication Date

January 1, 2021

Start / End Page

2161 / 2169
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Bagaria, A., Senthil, J., Slivinski, M., & Konidaris, G. (2021). Robustly Learning Composable Options in Deep Reinforcement Learning. In IJCAI International Joint Conference on Artificial Intelligence (pp. 2161–2169).
Bagaria, A., J. Senthil, M. Slivinski, and G. Konidaris. “Robustly Learning Composable Options in Deep Reinforcement Learning.” In IJCAI International Joint Conference on Artificial Intelligence, 2161–69, 2021.
Bagaria A, Senthil J, Slivinski M, Konidaris G. Robustly Learning Composable Options in Deep Reinforcement Learning. In: IJCAI International Joint Conference on Artificial Intelligence. 2021. p. 2161–9.
Bagaria, A., et al. “Robustly Learning Composable Options in Deep Reinforcement Learning.” IJCAI International Joint Conference on Artificial Intelligence, 2021, pp. 2161–69.
Bagaria A, Senthil J, Slivinski M, Konidaris G. Robustly Learning Composable Options in Deep Reinforcement Learning. IJCAI International Joint Conference on Artificial Intelligence. 2021. p. 2161–2169.

Published In

IJCAI International Joint Conference on Artificial Intelligence

ISSN

1045-0823

ISBN

9780999241196

Publication Date

January 1, 2021

Start / End Page

2161 / 2169