Optimal Lateral Path-Tracking Control of Vehicles with Partial Unknown Dynamics via DPG-Based Reinforcement Learning Methods

This article focuses on the optimal lateral path-tracking control problem of vehicles with unknown drift dynamics in a model-free manner through two novel deterministic policy gradient (DPG) based reinforcement learning (RL) methods. First, due to the difficulty of modeling the precise dynamics of vehicles, a policy gradient (PG) is derived to learn the optimal control gain by minimizing a predefined infinite-horizon performance index, where the knowledge of the system drift dynamics of vehicles is no longer needed. Then, to further remove the limitation of the initial admissibility of the control policy, a two-stage DPG-based RL optimal control algorithm is proposed, in which a novel finite-horizon performance index is employed in the pre-learning stage such that the control gain does not require to be initially admissible. It should be pointed out that the derived PGs in the two algorithms are based on an explicit form only using a single sampling data for each calculation rather than an estimated form via randomly perturbing feedback gains, which reduces the sampling and computational complexity of the algorithms. Finally, the simulations of the lateral path-tracking control of vehicles have verified the effectiveness and superiority of the proposed DPG-based RL algorithms compared with existing methods.