AoI-Aware Joint Sampling-Buffering-Routing Optimization for Autonomous UAV Swarms via a MARL Approach

Real-time monitoring in infrastructure-constrained environments presents critical challenges for time-sensitive applications. Conventional approaches relying on terrestrial sensors are impractical in remote regions, whereas UAV swarms operating as Flying Ad-hoc Networks (FANETs) offer autonomy but still depend on pre-deployed ground infrastructure. To overcome this limitation, we adopt a Leader–Follower UAV swarm architecture in which Follower UAVs serve dual roles as sensing platforms and communication relays, enabling a fully autonomous aerial monitoring system with enhanced adaptability. The effectiveness of real-time monitoring hinges on data freshness, rendering the optimization of information timeliness essential. To address this, we introduce the Adaptive Age-aware Sampling–Buffering–Routing (AASBR) framework to minimize Age of Information (AoI) through coordinated sampling, buffering, and routing. We model this problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) and propose a novel multi-agent reinforcement learning (MARL) approach named Curriculum Orchestrated Multi-head Multi-Agent Proximal Policy Optimization (COMH-MAPPO) algorithm. COMH-MAPPO employs a multi-head policy architecture combined with curriculum learning to progressively address coupled decision-making challenges under partial observability. Simulation results demonstrate that COMH-MAPPO achieves over 48% and 15% improvement in average AoI compared with MARL and ablation benchmarks, respectively, while also outperforming benchmarks across key network metrics including transmission latency, packet delivery ratio, and throughput.