Policy Extraction-Based Adversarial Attack in Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning (MARL) has achieved remarkable progress across diverse domains. Nevertheless, MARL-trained agents remain vulnerable to adversarial perturbations on their observations, which can severely degrade the overall team performance. In reality, it is hard to perform adversarial attacks on the agent team when the models and datasets of agents are unreachable by attackers. In this paper, we introduce the policy extraction-based adversarial attack in MARL, which provides methods for building substitute models to attack the agent team. We firstly collect the data of the victim agent by only observing its behaviors, and then train the substitute policy model with the collected dataset and multiple data augmentation methods. Finally, we perform a newly designed optimization-based black box adversarial attack on the victim agent with the stolen policy model. We conduct experiments on SMAC environments to carefully test the performance of the substitute policy models and attack effects of adversarial attacks. Experiment results demonstrate that our method could successfully change the actions of the agent in the agent team trained by QMIX algorithm in a black box environment.