Cooperatively Caching Mechanism in Large-Scale LEO Satellite Networks: An Age-Driven Multiagent Deep Reinforcement Learning Approach

In large-scale low Earth orbit (LEO) satellite networks, cache placement and update are facing challenges such as limited cache volume, update capacity, frequently interrupted intersatellite links (ISLs), and sudden changes in content popularity, leading to the difficulty for obtaining fresh data for existing algorithms by optimizing average user access delay instead of timeliness and adaptability. To address this issue, in this work, we propose an age-driven multiagent deep reinforcement learning (MADRL)-based cooperative cache and update mechanism called as age-driven cooperative cache and update (ADCCU) algorithm, in which we establish an age-driven cache gain function to evaluate the effective value of holding a data item from the cache's perspective. In particular, we formulate the age-driven cooperatively cache scheduling issue as an integer programming (IP) problem and solve it by exploiting a Markov decision process (MDP). The simulation results demonstrate that, under a cache capacity constraint of c_{i} = 3, the ADCCU algorithm significantly outperforms baseline caching strategies: the average cache gain is increased by approximately 1.09 compared to the deep deterministic policy gradient (DDPG)-based cache, 2.88 compared to the least frequently used (LFU), and 4.92 compared to the least recently used (LRU), respectively, and the cache hit ratio is improved by approximately 5.31% over the DDPG-based cache, 56.4% over LFU, and 65.9% over LRU, respectively.