Research on Intelligent Routing For Integrated Satellite-Terrestrial Networks Through Autonomous Multi-Agent Collaboration

Recent years have witnessed the construction plans of large, low Earth orbit (LEO) satellite constellations, such as Starlink, has gained significant interest and are expected to assume a critical role in integrated satellite-terrestrial networks as the foundation of the forthcoming 6G network. However, ensuring optimal network Quality of Service (QoS) performance presents significant challenges arising from the variability of inter-satellite and satellite-to-ground links, coupled with the uneven geographical distribution of ground service requests. A large body of current literature emphasizes inter-satellite routing with singular optimization objectives, proving insufficient for the requirements of integrated satellite-terrestrial networks. This study addresses these challenges by approaching integrated network routing as a two-pronged problem. First, it focuses on satellite-to-ground link selection in defined constraints. Second, it employs a multi-hop Markov decision process for inter-satellite links. Moreover, the study establishes predictive models for both satellite-to-ground and inter-satellite link channels and states. It also introduces a node congestion model that factors in the distribution of service requests. A novel routing framework for the integrated satellite-terrestrial network, based on an autonomous multi-agent collaboration system, has been designed. This framework incorporates an enhanced reward strategy derived from cooperative game theory. In this framework, each satellite node acts as an agent, executing forwarding decisions through actor-critic deep reinforcement learning (AC-DRL). Dynamic reward adjustments based on the network state optimize data forwarding decision-making. The results demonstrate that while this research achieves similar path length performance to established routing methods such as shortest path search, it derives significant improvements in network QoS performance. Specifically, it enhances latency, jitter, packet loss rate, and bandwidth.