Skip to main content

Extracting Latent State Representations with Linear Dynamics from Rich Observations

Publication ,  Conference
Frandsen, A; Ge, R; Lee, H
Published in: Proceedings of Machine Learning Research
January 1, 2022

Recently, many reinforcement learning techniques have been shown to have provable guarantees in the simple case of linear dynamics, especially in problems like linear quadratic regulators. However, in practice many tasks require learning a policy from rich, high-dimensional features such as images, which are unlikely to be linear. We consider a setting where there is a hidden linear subspace of the high-dimensional feature space in which the dynamics are linear. We design natural objectives based on forward and inverse dynamics models. We prove that these objectives can be efficiently optimized and their local optimizers extract the hidden linear subspace. We empirically verify our theoretical results with synthetic data and explore the effectiveness of our approach (generalized to nonlinear settings) in simple control tasks with rich observations.

Duke Scholars

Published In

Proceedings of Machine Learning Research

EISSN

2640-3498

Publication Date

January 1, 2022

Volume

162

Start / End Page

6705 / 6725
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Frandsen, A., Ge, R., & Lee, H. (2022). Extracting Latent State Representations with Linear Dynamics from Rich Observations. In Proceedings of Machine Learning Research (Vol. 162, pp. 6705–6725).
Frandsen, A., R. Ge, and H. Lee. “Extracting Latent State Representations with Linear Dynamics from Rich Observations.” In Proceedings of Machine Learning Research, 162:6705–25, 2022.
Frandsen A, Ge R, Lee H. Extracting Latent State Representations with Linear Dynamics from Rich Observations. In: Proceedings of Machine Learning Research. 2022. p. 6705–25.
Frandsen, A., et al. “Extracting Latent State Representations with Linear Dynamics from Rich Observations.” Proceedings of Machine Learning Research, vol. 162, 2022, pp. 6705–25.
Frandsen A, Ge R, Lee H. Extracting Latent State Representations with Linear Dynamics from Rich Observations. Proceedings of Machine Learning Research. 2022. p. 6705–6725.

Published In

Proceedings of Machine Learning Research

EISSN

2640-3498

Publication Date

January 1, 2022

Volume

162

Start / End Page

6705 / 6725