Adaptive dynamic programming for unknown continuous-time nonlinear systems via bias-policy iteration

In this paper, a bias-policy iteration (Bias-PI) method is proposed to relax the requirement of the policy iteration method on the initial admissible control and achieve optimal control for unknown continuous-time nonlinear systems. First, a model-based Bias-PI method is introduced that uses a bias value function to ease the constraints of the initial admissible control. The boundedness of the bias value function and the convergence of the algorithm are demonstrated through rigorous mathematical proofs. Further, the data-driven implementation of the Bias-PI method is detailed, highlighting its ability to learn an optimal controller without prior system information, and simultaneously retaining the fast convergence properties of the traditional policy iteration algorithm. The effectiveness of the data-driven Bias-PI method is illustrated through two simulation examples.