Single Timescale Actor-Critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees
Publication
, Journal Article
Zhou, M; Lu, J
Published in: Journal of Machine Learning Research
January 1, 2023
We propose a single timescale actor-critic algorithm to solve the linear quadratic regulator (LQR) problem. A least squares temporal difference (LSTD) method is applied to the critic and a natural policy gradient method is used for the actor. We give a proof of convergence with sample complexity O(ε−1 log(ε−1)2). The method in the proof is applicable to general single timescale bilevel optimization problems. We also numerically validate our theoretical results on the convergence.
Duke Scholars
Published In
Journal of Machine Learning Research
EISSN
1533-7928
ISSN
1532-4435
Publication Date
January 1, 2023
Volume
24
Related Subject Headings
- Artificial Intelligence & Image Processing
- 4905 Statistics
- 4611 Machine learning
- 17 Psychology and Cognitive Sciences
- 08 Information and Computing Sciences
Citation
APA
Chicago
ICMJE
MLA
NLM
Zhou, M., & Lu, J. (2023). Single Timescale Actor-Critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees. Journal of Machine Learning Research, 24.
Zhou, M., and J. Lu. “Single Timescale Actor-Critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees.” Journal of Machine Learning Research 24 (January 1, 2023).
Zhou M, Lu J. Single Timescale Actor-Critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees. Journal of Machine Learning Research. 2023 Jan 1;24.
Zhou, M., and J. Lu. “Single Timescale Actor-Critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees.” Journal of Machine Learning Research, vol. 24, Jan. 2023.
Zhou M, Lu J. Single Timescale Actor-Critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees. Journal of Machine Learning Research. 2023 Jan 1;24.
Published In
Journal of Machine Learning Research
EISSN
1533-7928
ISSN
1532-4435
Publication Date
January 1, 2023
Volume
24
Related Subject Headings
- Artificial Intelligence & Image Processing
- 4905 Statistics
- 4611 Machine learning
- 17 Psychology and Cognitive Sciences
- 08 Information and Computing Sciences