SFOFusion: a task-oriented meta-learning framework for spatial–frequency fusion of infrared and visible images

Infrared and visible image fusion plays a crucial role in various applications such as computer vision and remote sensing, especially in low-visibility environments, where it combines the powerful perception capability of infrared images in low light with the rich detail information of visible images. However, existing methods typically process image features separately in either the spatial or frequency domain, neglecting the deep fusion and optimization of cross-domain information. Moreover, few methods consider how to jointly optimize downstream tasks with spatial-frequency fusion models. To address these issues, we propose a task-oriented meta-learning framework, SFOFusion, which integrates phase information from the frequency domain with an explicit difference compensation mechanism in the spatial domain. The frequency domain part enhances cross-domain fusion accuracy by progressively merging amplitude and phase information, while the spatial domain retains high-frequency information through explicit compensation strategies. The training strategy uses an internal–external bi-level meta-learning scheme with epoch-level dynamic balancing. The internal–external loops introduce downstream task feedback into the fusion-loss optimization process, while the epoch-level balancing mechanism adaptively adjusts the relative importance of fusion fidelity and task performance to prevent either objective from dominating training. Experimental results demonstrate that SFOFusion significantly improves fusion performance on multiple datasets while maintaining model lightweight. It achieves a 4.7% improvement in mAP@0.5 for object detection and a 6.3% improvement in mIoU for semantic segmentation. Furthermore, SFOFusion reduces computational demands, with 25% fewer parameters and 30% less computation compared to traditional methods, highlighting its efficiency and practicality. The code is available at: https://github.com/Simon-JT/SFOFusion.