Robustness-Guaranteed Reinforcement Learning under Uncertainties in Dynamics Modeling and State Estimates

Robust reinforcement learning enhances system robustness against uncertainties through bilevel optimization. However, the controller learned by reinforcement learning struggles to offer strict robustness guarantee, limiting its reliability in realworld applications. This paper presents robustness-guaranteed reinforcement learning framework under bounded uncertainties in dynamics modeling and state estimates. By describing the uncertainties and system dynamics models via ReLU neural networks, the states that most violate the proposed robustness conditions can be exactly found, and their robustness is prioritized during training. The solvability of the reinforcement learning problem is particularly studied, yielding a necessary condition that captures the adversarial relationship between control saturation and uncertainties. Subsequently, the attractive region can be pre-validated and the maximum robustness of the controller can be achieved by amplifying the uncertainties to a derived maximum extent in training. The proposed method has been fully evaluated in simulation and its advantages have been proven, including 3D quadrotor reference trajectory tracking tasks.