Navigation for autonomous vehicles via fast-stable and smooth reinforcement learning

This paper investigates the navigation problem of autonomous vehicles based on reinforcement learning (RL) with both stability and smoothness guarantees. By introducing a data-based Lyapunov function, the stability criterion in mean cost is obtained, where the Lyapunov function has a property of fast descending. Then, an off-policy RL algorithm is proposed to train safe policies, in which a more strict constraint is exerted in the framework of model-free RL to ensure the fast convergence of policy generation, in contrast with the existing RL merely with stability guarantee. In addition, by simultaneously introducing constraints on action increments and action distribution variations, the difference between the adjacent actions is effectively alleviated to ensure the smoothness of the obtained policy, instead of only seeking the similarity of the distributions of adjacent actions as commonly done in the past literature. A navigation task of a ground differentially driven mobile vehicle in simulations is adopted to demonstrate the superiority of the proposed algorithm on the fast stability and smoothness.