A Graph-Enhanced MLLM for Hierarchical Multimodal Emotion Understanding and Support in Conversations

Multimodal emotion dialogue system involves a hierarchical progression: Perception (Multimodal Emotion Recognition in Conversation, MERC), Reasoning (Multimodal Emotion-Cause Pair Extraction, MECPE), and Support (Multimodal Emotional Support Conversation, MESC). Although existing methods have shifted toward MLLM-based paradigms with stronger generalization ability, they still handle MERC, MECPE, and MESC as independent components. This decoupled design disrupts the hierarchical relationship among perception, reasoning, and support. Moreover, their multimodal fusion relies on coarse-grained concatenation of encoder features into the prompt, ignoring structured relational dependencies among speakers and modalities. To address these limitations, we propose GraphEMO, a unified multimodal emotion understanding and support framework based on a graph-enhanced MLLM. It models MERC, MECPE, and MESC as a residual chain of task-specific priors, enabling MESC to build on perceptual and causal representations from preceding stages. In addition, we leverages dual graph attention to capture cross- and intra-modal relational dynamics among speakers, enhancing structured relational modeling. Experiments on MERC, MECPE, and MESC benchmarks show that GraphEMO consistently improves performance on Qwen and LLaMA models, achieving state-of-the-art results.