Motion control of unmanned surface vehicle based on improved reinforcement learning proximal policy optimization algorithm

Shuai Wu

doi:10.1117/12.2653441

27 September 2022 Motion control of unmanned surface vehicle based on improved reinforcement learning proximal policy optimization algorithm

Shuai Wu

Author Affiliations +

Proceedings Volume 12346, 2nd International Conference on Information Technology and Intelligent Control (CITIC 2022); 1234615 (2022) https://doi.org/10.1117/12.2653441
Event: 2nd International Conference on Information Technology and Intelligent Control (CITIC 2022), 2022, Kunming, China

Abstract

In this paper, a novel deep reinforcement learning algorithm Proximal Policy Optimization (PPO) based on F-divergence is proposed to realize the motion control of unmanned surface vehicle. Aiming at the nonlinear and underactuated characteristics of unmanned surface vehicle system, the new reinforcement learning algorithm can overcome the problem that PPO algorithm falls into local optimization in the training process, and improve the diversity of algorithm exploration. Based on the Open AI simulation environment, this paper analyzes the motion law of the unmanned surface vehicle on the water surface, and establishes a three-degree of freedom kinematics and dynamics mathematical model. An improved reinforcement learning algorithm is used to design the motion controller of unmanned surface vehicle, and a compound reward is designed, which effectively improves the learning efficiency of the network. Simulation results shows the effectiveness of the improved Proximal Policy Optimization algorithm in the motion control of unmanned surface vehicle, and verify the superiority of the improved algorithm.

Citation Download Citation

Shuai Wu "Motion control of unmanned surface vehicle based on improved reinforcement learning proximal policy optimization algorithm", Proc. SPIE 12346, 2nd International Conference on Information Technology and Intelligent Control (CITIC 2022), 1234615 (27 September 2022); https://doi.org/10.1117/12.2653441

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available