Abstract
Multimodal Sentiment Analysis (MSA) with uncertain missing modalities poses a new challenge in sentiment analysis. To address this problem, existing methods often suffer from either ineffective fusion strategies for complex interactions or a lack of robustness when modality data is incomplete. In this paper, we propose a novel framework, RMSAGF, which features a synergistic dual-component architecture designed to address these issues systematically. First, for complete data, an Interdependent Multimodal Attention (IMA) module is designed for deep, iterative fusion to capture comprehensive inter- and intra-modal dynamics. Second, to handle missing modalities, a Context-Aware Feature Generation (CFG) module introduces an active generation paradigm. When a modality is missing, the CFG module first dynamically assesses the reliability of the present modalities to guide a dual-level generation strategy. It performs rapid utterance-level compensation while simultaneously generating a complete, temporally coherent feature sequence for the missing modality via a context-aware Transformer decoder. Extensive experiments on the CMU-MOSI, CMU-MOSEI, and CH-SIMS benchmarks demonstrate that RMSAGF establishes a new state of the art, particularly on core regression metrics under incomplete settings, thereby validating the superiority of its active generation paradigm.
| Original language | English |
|---|---|
| Article number | 131841 |
| Journal | Expert Systems with Applications |
| Volume | 316 |
| DOIs | |
| State | Accepted/In press - 2026 |
| Externally published | Yes |
Keywords
- Context-aware generation
- Cross-attention
- Dynamic adaptation
- Interdependent attention
- Multimodal sentiment analysis
- Transformer decoders
Fingerprint
Dive into the research topics of 'RMSAGF: A synergistic generation and fusion framework for robust multimodal sentiment analysis'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver