Approximate dynamic programming recurrence relations for a hybrid optimal control problem


Conference Paper

This paper presents a hybrid approximate dynamic programming (ADP) method for a hybrid dynamic system (HDS) optimal control problem, that occurs in many complex unmanned systems which are implemented via a hybrid architecture, regarding robot modes or the complex environment. The HDS considered in this paper is characterized by a well-known three-layer hybrid framework, which includes a discrete event controller layer, a discrete-continuous interface layer, and a continuous state layer. The hybrid optimal control problem (HOCP) is to nd the optimal discrete event decisions and the optimal continuous controls subject to a deterministic minimization of a scalar function regarding the system state and control over time. Due to the uncertainty of environment and complexity of the HOCP, the cost-to-go cannot be evaluated before the HDS explores the entire system state space; as a result, the optimal control, neither continuous nor discrete, is not available ahead of time. Therefore, ADP is adopted to learn the optimal control while the HDS is exploring the environment, because of the online advantage of ADP method. Furthermore, ADP can break the curses of dimensionality which other optimizing methods, such as dynamic programming (DP) and Markov decision process (MDP), are facing due to the high dimensions of HOCP. © 2012 Copyright Society of Photo-Optical Instrumentation Engineers (SPIE).

Full Text

Duke Authors

Cited Authors

  • Lu, W; Ferrari, S; Fierro, R; Wettergren, TA

Published Date

  • December 1, 2012

Published In

Volume / Issue

  • 8387 /

International Standard Serial Number (ISSN)

  • 0277-786X

International Standard Book Number 13 (ISBN-13)

  • 9780819490650

Digital Object Identifier (DOI)

  • 10.1117/12.919286

Citation Source

  • Scopus