Temporal Information and Feature Fusion for RGBT Tracking

Robust multimodal sensing is essential for reliable object tracking in complex environments. Although RGB-thermal (RGBT) sensing leverages complementary spectral modalities to enhance tracking performance, existing methods often rely on static template matching, thereby neglecting the inherent temporal continuity of sensor streams. This limitation compromises tracking stability under challenging sensing conditions, such as abrupt illumination changes, severe occlusions, and thermal crossover. To address these challenges, this article proposes TFTrack, a new type of closed-loop video-stream RGBT tracking framework that has adopted a deep-collaborative two-component time design: a dynamic template library, which provides spatial priors for trajectory modeling by storing the prediction box results of historical templates; and autoregressive time label propagation, which compensates for the continuity deficiency of pure template methods. Unlike traditional sparse sampling paradigms, TFTrack exploits continuous temporal informatics by integrating historical appearance features and motion trajectories within a dynamic template library, ensuring consistent perception across time. Furthermore, to maximize multisensor synergy, we develop an input-feature-guided attention-fusion (IFGAFusion) module. By dynamically recalibrating spatial importance weights under the guidance of input features, this module effectively fuses heterogeneous data streams from RGB and thermal infrared (TIR) sensors. Extensive qualitative and quantitative experiments demonstrate that the proposed method achieves state-of-the-art performance. Notably, TFTrack operates at 30 FPS on the LasHeR benchmark with a precision rate of 74.7% and a success rate of 59.6%, satisfying the stringent real-time and reliability requirements for practical vision-based sensing systems.