Age of Information optimization with Heterogeneous UAVs based on Deep Reinforcement Learning

Recent years have witnessed increasingly more Unmanned Aerial Vehicle (UAV) applications for data collection in the Internet of Things (IoT). Due to the limited energy and service capacity, it is very challenging for a single UAV to accomplish the data collection while guaranteeing the information freshness of IoT devices or sensor nodes (SNs). In practice, different types of UAVs may have different energy capabilities. In this paper, we propose a more practical heterogeneous UAV swarm path planning problem for optimizing the information freshness, in which the division and cooperation among UAVs with different energy capacities have been taken into consideration. The freshness, i.e., age of information (AoI) collected from each SN is characterized by the data uploading time and the time elapsed since the UAV leaves this SN. We successfully present a deep reinforcement learning algorithm based on attention mechanism by end-to-end training to optimize the average age under UAVs' energy constraints. The simulation results show that our algorithm has fast convergence, high optimization capability and reliability, and can solve the heterogeneous UAV swarm cooperative AoI optimization problem effectively.