Distributed optimal consensus control for discrete-time multi-agent systems with composite switching topologies via an iterative Q-learning method

This paper aims to achieve distributed optimal consensus control for discrete-time multi-agent systems (MASs) under composite switching topologies through utilising the iterative Q-learning approach. First, the optimal consensus control strategy, independent of system dynamics information, is developed by using the designed iterative Q-learning approach to guarantee that all follower states can synchronise to the desired trajectory formulated by the leader. Compared with the invariable algebraic or edge-fixed topology structures, the designed Q-learning control algorithm in this paper is implemented based on the composite switching topologies consisting of periodic and stochastic mechanisms. Then, an error dynamic system is established, which transforms the optimal consensus control issue into an optimal regulation issue and removes the influence of the two types of switching topologies. To conquer the problems of lack of model information and the inaccessible analytical solution of Hamilton-Jacobi-Isaacs (HJI) equation, an iterative Q-learning approach is designed according to the reconstructed Q-function Bellman equation. Through further iterative learning, the optimal control strategy can be acquired so as to realise consensus control and ensure the stability of the closed-loop systems. Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed distributed optimal control scheme.