A Collaborative Control Method for Spacecraft Clusters Based on Multi Agent Reinforcement Learning

Cluster intelligence refers to the emergence of collective behavior, such as collaborative detection, which compensates for individual limitations and accomplishes complex tasks through effective coordination among intelligent agents. The distributed strategy necessitates high autonomy for each spacecraft, with communication connections between adjacent spacecraft enabling state exchange. Firstly, the composition of the spacecraft cluster detection system described in this article is introduced. Then, a multi-agent reinforcement learning algorithm is introduced to address the aforementioned multivariable sequence decision-making problem. The entire sequence decision problem is divided into multiple time steps for multi-agent reinforcement learning modeling. Agents interact with the environment and receive reward feedback from it. After adopting the Actor Critic algorithm, each agent's optimization goal is to maximize their cumulative expected reward. The actor aims to learn the agent's strategy function and maximize expected cumulative rewards, while critics learn a value function to evaluate current state value and guide actor strategy optimization. Finally, scenario design rules and reward settings are based on collaborative target detection by search and tracking spacecrafts. This enables collaborative control of both types of spacecrafts, achieving 35 successful target tracks in line with task requirements.