Saliency-Aware Adversarial Attacks on Visual Trackers

Object tracking is a fundamental task in computer vision, and ensuring its security is crucial for reliable and safe real-world applications. Similar background distractors are commonly present in tracking scenarios and often result in tracking failures. However, tracking attack methods leveraging background similarity have yet to receive adequate attention. In this paper, we propose a saliency-aware guided adversarial attack method for tracking models. The proposed method aims to evaluate the performance of advanced trackers under adversarial attacks, targeting interference from similar objects to reveal their vulnerabilities. Specifically, we introduce a text-guided similar region localization module that leverages semantic features of coarse-grained category attributes to select regions from multiple frames that are both visually and semantically similar to the target, overcoming the limitations of random selection methods. In addition, we develop a saliency-aware search region feature constraint module that incorporates foreground-background saliency information. By dynamically filtering feature weights through the joint use of fine-grained target appearance and text descriptions, this adaptive mechanism reduces the confidence of the tracker in foreground-background distinctions, progressively leading to target drift. We conduct extensive experiments on multiple benchmark datasets over multiple types of trackers, and the results demonstrate that the proposed method can effectively attack existing trackers and reveal their potential security risks.