HPFA: A hybrid perception and feature attention framework for multimodal sentiment analysis with missing modalities

Multimodal Sentiment Analysis (MSA) predominantly assumes the availability of complete multimodal data. However, in real-world scenarios, missing modalities are inevitable due to sensor errors or data corruption. Conventional methods relying on complete-data assumptions suffer from significant performance degradation and reduced robustness under these conditions. To address this challenge, we propose a Hybrid Perception and Feature Attention (HPFA) framework. Drawing inspiration from human cognitive mechanisms for handling incomplete information, this framework incorporates two primary modules. The first, DynaSynapse Fusion (DSF), adopts a modality-specific strategy that prioritizes interactions between the language modality and auxiliary (visual and audio) modalities, followed by unimodal enhancement to maximize the utility of available features. The second, the Cyclic Cross-Modal Recoverer (CCR), progressively reconstructs missing modalities via three cascaded reconstruction units. Each unit comprises a Transformer layer to capture global dependencies among multimodal features and a Spatial-Updating and Normalization (SUN) module to facilitate fine-grained cross-modal interactions, thereby enabling stepwise reconstruction of missing modality information. Extensive experiments on three public benchmarks demonstrate that HPFA consistently achieves superior performance and robustness across various missing data conditions, establishing it as an effective solution for MSA in realistic, incomplete-data scenarios.