Data-Driven Bias-Lyapunov Iteration for Optimal Control of Unknown Markovian Jump Linear Systems

In this paper, a bias-Lyapunov iteration method is proposed to solve the optimal control problem of unknown Markovian jump linear systems. By incorporating a bias parameter into the conventional Lyapunov iteration method, the proposed method eliminates initial admissible control requirements. A model-based theoretical framework is subsequently established, accompanied by a rigorous convergence proof for the modified iteration process. Subsequently, a data-driven version of bias-Lyapunov iteration is developed to learn an optimal control for Markovian jump linear systems with completely unknown dynamics. Simulation examples validate the efficacy and advantage of the proposed bias-Lyapunov iteration method.