FGLight: Learning Neighbor-level Information for Traffic Signal Control

In recent years, multi-agent reinforcement learning (MARL) methods have increasingly been applied to traffic signal control and have achieved some success. However, most of existing MARL methods often underemphasize the heterogeneity in neighborhood-level information of the same agent. This results in highly sensitive performances and a long learning process. To address this challenge, we propose FGLight, a novel Feudal MARL method for traffic signal control. FGLight leverages Adaptive Graph Attention Networks (AGAT) to dynamically model the interactive relationships between intersections. Through adaptive neighbor selection and weight-based attention mechanisms, AGAT dynamically assigns importance weights to neighbor-level information, thereby improving the accuracy of local policies by more effectively exploiting neighborhood information. Moreover, FGLight introduces a Smooth Hysteretic Deep Q-Network (SHDQN) based on an optimistic assumption mechanism, which enhances the stability of the global policy. We conducted experiments on both synthetic and real-world datasets, and the results demonstrate that, compared to several state-of-the-art MARL methods, FGLight performs better as the complexity of the road network increases, exhibiting faster convergence and greater policy stability.