SHL-Net: Semantics-Enhanced Network for Localizing Harmonized Image Splicing

Recent advancements in image harmonization techniques have made it increasingly easy to seamlessly composite images, facilitating the creation of highly convincing fake images. As a result, the detection and localization of such high-confidence manipulations has become a critical challenge in image forensics. However, existing image manipulation localization methods often struggle to maintain performance when identifying this specific type of manipulation. In response to this limitation, we propose a novel model called the Semantics-Enhanced Harmonization Localization Network (SHL-Net) to robustly and accurately localize such manipulations. Specifically, SHL-Net consists of two sub-networks: the Splicing Trace Localization Network (STLNet) and the Localization Enhancement Network (LENet). STLNet is responsible for extracting forgery-relevant information and generating a coarse mask of the harmonized regions. LENet then leverages semantic prior knowledge from the pretrained Segment Anything Model (SAM) to refine the coarse mask into complete semantic units. To enhance the adaptation of SAM for image harmonization localization, we devise a novel Forgery-Aware SAM Adapter (FASA), which integrates semantic prior knowledge with tampering characteristics. Additionally, we design a Multi-View Feature Fusion (MVFF) module to adaptively fuse multi-view features from different stages, further improving the model's accuracy. Extensive experimental results demonstrate the superior performance of SHL-Net over other state-of-the-art localization methods in the image harmonization localization task. The source code is available at https://github.com/HITFuxiwen/SHL-Net