Approximate dynamic programming recurrence relations for a hybrid optimal control problem

W. Lu; S. Ferrari; R. Fierro; T. A. Wettergren

doi:10.1117/12.919286

17 May 2012 Approximate dynamic programming recurrence relations for a hybrid optimal control problem

W. Lu, S. Ferrari, R. Fierro, T. A. Wettergren

Proceedings Volume 8387, Unmanned Systems Technology XIV; 83870C (2012) https://doi.org/10.1117/12.919286
Event: SPIE Defense, Security, and Sensing, 2012, Baltimore, Maryland, United States

Abstract

This paper presents a hybrid approximate dynamic programming (ADP) method for a hybrid dynamic system (HDS) optimal control problem, that occurs in many complex unmanned systems which are implemented via a hybrid architecture, regarding robot modes or the complex environment. The HDS considered in this paper is characterized by a well-known three-layer hybrid framework, which includes a discrete event controller layer, a discrete-continuous interface layer, and a continuous state layer. The hybrid optimal control problem (HOCP) is to nd the optimal discrete event decisions and the optimal continuous controls subject to a deterministic minimization of a scalar function regarding the system state and control over time. Due to the uncertainty of environment and complexity of the HOCP, the cost-to-go cannot be evaluated before the HDS explores the entire system state space; as a result, the optimal control, neither continuous nor discrete, is not available ahead of time. Therefore, ADP is adopted to learn the optimal control while the HDS is exploring the environment, because of the online advantage of ADP method. Furthermore, ADP can break the curses of dimensionality which other optimizing methods, such as dynamic programming (DP) and Markov decision process (MDP), are facing due to the high dimensions of HOCP.

Citation Download Citation

W. Lu, S. Ferrari, R. Fierro, and T. A. Wettergren "Approximate dynamic programming recurrence relations for a hybrid optimal control problem", Proc. SPIE 8387, Unmanned Systems Technology XIV, 83870C (17 May 2012); https://doi.org/10.1117/12.919286

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available