Multi-UAV Cooperative Search in Multi-Layered Aerial Computing Networks: A Multi-Agent Deep Reinforcement Learning Approach

Multi-UAV Cooperative Search (MCS) can significantly enhance the efficiency and effectiveness of search by enabling multiple unmanned aerial vehicles (UAVs) to collaborate in conducting search missions. Thus, it has played a vital role in various applications, such as surveillance, target detection, and information gathering. While existing works in this field mainly focused on a single UAV layer, in this work we consider a Multi-layered Aerial Computing Network (MACN) scenario, which consists of a Low-Altitude Platform (LAP) layer with multiple high-flexibility and low-capacity UAVs (called LUAVs) and a High-Altitude Platform (HAP) layer with one low-flexibility and high-capacity UAV (called HUAV). In such a scenario, We focus on the joint optimization of flying trajectories, computation offloading, and resource allocation, aiming at minimizing the uncertainty of Search Probability Map (SPM). The problem is challenging due to the co-existence of discrete and continuous decision variables, as well as the fast and randomly changing of wireless environment. To solve the problem in an online and distributed manner, we propose a Multi-Agent Deep Reinforcement Learning (MADRL) approach based on Parameter Sharing and Action Mask (PSAM), called PSAMMA, where the State-Action-Reward-State-Action (SARSA) method is leveraged to determine the discrete flying and offloading decisions. Experiment results show that the proposed PSAMMA algorithm outperforms existing methods in terms of the average SPM uncertainty, the target discovery rate, and the coverage rate.