AoI-Aware Scheduling and Resource Control via Hierarchical DRL in Beam Hopping LEO Satellite

Low Earth Orbit (LEO) satellite networks have become a promising solution to support Internet of Things (IoT) services, especially in remote or infrastructure-limited areas. In LEO-enabled IoT scenarios, timely data delivery is essential for situational awareness and real-time decision-making. To measure the freshness of the data, Age of Information (AoI) has been widely adopted as a key performance metric, particularly in applications with frequent status updates. However, minimizing AoI in LEO systems is challenging due to limited onboard resources and dynamic traffic demands. This paper focuses on the joint optimization of beam hopping (BH) scheduling and resource block (RB) allocation to reduce the average AoI across the network. To handle the complex problem, we propose a hierarchical deep reinforcement learning (DRL) framework. At the high level, a centralized satellite agent leverages the Proximal Policy Optimization (PPO) algorithm to determine the beam illumination pattern over the service area. At the low level, a decentralized multi-agent PPO framework is employed, where each beam-level agent independently allocates resource block (RB) to users, aiming to improve data freshness and transmission efficiency. Simulation results show that the proposed method outperforms three benchmark strategies in reducing average AoI.