A Multiagent Deep Reinforcement Learning Approach for Multi-UAV Cooperative Search in Multilayered Aerial Computing Networks

Multi-UAV cooperative search (MCS) can significantly enhance the efficiency and effectiveness of search by enabling multiple unmanned aerial vehicles (UAVs) to collaborate in conducting search missions. Thus, it has played a vital role in various applications, such as surveillance, target detection, and information gathering. While existing works in this field mainly focused on a single UAV layer, in this work we consider a multilayered aerial computing network (MACN) scenario, which consists of a low-altitude platform (LAP) layer with multiple high-flexibility and low-capacity UAVs (called LUAVs) and a high-altitude platform (HAP) layer with one low-flexibility and high-capacity UAV (called HUAV). In such a scenario, We focus on the joint optimization of flying trajectories, computation offloading, and resource allocation, aiming at minimizing the uncertainty of search probability map (SPM), and meanwhile maximizing the number of target discovery and coverage rate. The problem is challenging due to the co-existence of discrete and continuous decision variables, as well as the fast and randomly changing of wireless environment. To solve the problem in an online and distributed manner, we propose a multiagent deep reinforcement learning (MADRL) approach based on the parameter sharing and action mask (PSAM), called PSAMMA, where the state-action-reward-state-action (SARSA) method is leveraged to determine the discrete flying and offloading decisions. Experiment results show that 1) the proposed PSAMMA algorithm outperforms existing algorithms in the literature, and can increase the average utility by 9.89%-31.15% and 2) we evaluate the search performance by analyzing the average uncertainty, target rate, and coverage rate under different parameter settings.