Efficient Communications for Multi-Agent Reinforcement Learning in Wireless Networks

Multi-agent reinforcement learning (RL) utilizes the observations and learning experiences shared among the agents to accelerate learning speed under partial observations and the resulting learning efficiency depends on the cooperative agent selection and the RL task state formulation. In this paper, we propose an efficient communication scheme for multi-agent RL that enables each learning agent to optimize the cooperative agent selection and the task state formulation to improve the learning performance and the quality of service for RL-based applications in wireless networks. Based on the local observation, the radio channel states, the similarity of RL task with neighboring agents and previous communication cost, this scheme formulates a communication state, which is input to a neural network to estimate the communication policy distribution. The RL task state of the learning agent, which consists of the local observation such as channel states and previous task performance, as well as the correlation between the shared and the local observation extracted based on the attention mechanism, is formulated to enhance the agent receptive field. In addition, the shared learning information is also exploited to update the local learning parameters such as the task Q-values and neural network weights and further improve the RL task policy exploration. As a case study, the proposed communication scheme is implemented in the multi-agent deep Q-network based anti-jamming unmanned aerial vehicle swarm communications and the performance gain over the benchmark is verified via simulation results.