TY - GEN
T1 - Cross-Modal Causal Scheduling for Enhancing Target-Oriented Multi-modal Sentiment Classification
AU - Zhao, Pengyu
AU - Li, Chaoyang
AU - Wang, Lingzhi
AU - Liao, Qing
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.
PY - 2025/10/4
Y1 - 2025/10/4
N2 - Target-oriented multi-modal sentiment classification (TMSC) aims to identify sentiment polarity towards specific targets by considering multiple modalities, e.g., text and images. However, current methods often ignore spurious correlations within the data, which can cause models to learn irrelevant features that misrepresent the sentiment of targets. To address this issue, we propose a novel Cross-Modal Causal Scheduling framework (CMCS) that prioritizes learning multi-modal features with fewer spurious correlations. Specifically, we first design a Multi-modal Feature Selection model (MFS) that utilizes causal intervention to select relevant features. Second, we construct a Causal cross-Modal Scheduler (CMS) to assess the causal effects of selected features, which further optimize the multi-modal learning process based on these effects. Finally, we formulate the CMS and the multi-modal learning process as a bi-level optimization problem. In the lower optimization, the MFS is updated with the scheduled gradient, while in the upper optimization, the CMS is updated with the implicit gradient. Extensive experiments demonstrate that our method outperforms existing baseline methods on TMSC and can effectively schedule the learning process of multi-modal features based on causal effects.
AB - Target-oriented multi-modal sentiment classification (TMSC) aims to identify sentiment polarity towards specific targets by considering multiple modalities, e.g., text and images. However, current methods often ignore spurious correlations within the data, which can cause models to learn irrelevant features that misrepresent the sentiment of targets. To address this issue, we propose a novel Cross-Modal Causal Scheduling framework (CMCS) that prioritizes learning multi-modal features with fewer spurious correlations. Specifically, we first design a Multi-modal Feature Selection model (MFS) that utilizes causal intervention to select relevant features. Second, we construct a Causal cross-Modal Scheduler (CMS) to assess the causal effects of selected features, which further optimize the multi-modal learning process based on these effects. Finally, we formulate the CMS and the multi-modal learning process as a bi-level optimization problem. In the lower optimization, the MFS is updated with the scheduled gradient, while in the upper optimization, the CMS is updated with the implicit gradient. Extensive experiments demonstrate that our method outperforms existing baseline methods on TMSC and can effectively schedule the learning process of multi-modal features based on causal effects.
KW - Causal Inference
KW - Multi-modal Sentiment Analysis
KW - Target-oriented multi-modal sentiment classification
UR - https://www.scopus.com/pages/publications/105019045491
U2 - 10.1007/978-3-032-06078-5_31
DO - 10.1007/978-3-032-06078-5_31
M3 - 会议稿件
AN - SCOPUS:105019045491
SN - 9783032060778
T3 - Lecture Notes in Computer Science
SP - 542
EP - 558
BT - Machine Learning and Knowledge Discovery in Databases. Research Track - European Conference, ECML PKDD 2025, Proceedings
A2 - Ribeiro, Rita P.
A2 - Pfahringer, Bernhard
A2 - Japkowicz, Nathalie
A2 - Larrañaga, Pedro
A2 - Jorge, Alípio M.
A2 - Soares, Carlos
A2 - Abreu, Pedro H.
A2 - Gama, João
PB - Springer Science and Business Media Deutschland GmbH
T2 - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2025
Y2 - 15 September 2025 through 19 September 2025
ER -