RoMATer: An end-to-end robust multiaircraft tracker with transformer

Multiple aircraft tracking (MAT) plays a critical role in military and civil aerial surveillance systems. Many studies have focused on tracking multiple pedestrians and automobiles, leaving a gap in related research on MAT because of the peculiar tracking properties of multiple aircraft, such as small/tiny object formation properties, identical shapes and appearances, severe camera jitter and trail cloud occlusion. In this paper, to fill this gap, building on top of a tracker named TransTrack, we present a robust MAT method to address multiple aircraft tracking, referred to as RoMATer. The improvements are threefold: a receptive field enlarging (RFE) module is first integrated into the backbone of a feature extractor to assist in feature extraction of full-scale aircraft, and a context-aware encoder layer (CaEL) is proposed to introduce global and local contextual embeddings to provide supplementary discriminative tracking information; moreover, a motion and appearance association (MA-A) module is proposed to overcome the tracking challenges of aircrafts possessing highly identical shapes and appearances. Extensive experiments on our established HIT-MATD (MAT Dataset) dataset (the first multiaircraft tracking dataset) verify the SOTA performance of RoMATer on multiaircraft tracking, with an increase of ~10% in terms of the MOTAL and an increase of ~5% with respect to the recall compared to that of TransTrack. Experiments on public RarePlanes dataset verify the effectiveness of the proposed modules in detecting multiple aircraft at full scales. Moreover, RoMATer can also run at a high frame rate (∼11FPS, Nvidia-A100).