ViSP: A PPO-enhanced framework for multimodal sarcasm generation with contrastive learning

Human emotions are inherently complex, with sarcasm representing one of the most subtle and distinctive forms. Although significant progress has been made in sarcasm understanding, sarcasm generation remains underexplored, largely due to the overreliance on textual modalities, the neglect of visual cues, and the semantic mismatch between images and sarcastic intent in existing datasets. In this work, we present M2SaG, a multimodal sarcasm generation dataset comprising 4970 samples, each containing an image, a sarcastic text, and its corresponding sarcasm target. To benchmark M2SaG, we propose ViSP, a ViLT-based sarcasm generation framework that integrates Proximal Policy Optimization (PPO) with contrastive learning. PPO leverages reward scores derived from DIP to guide the generation process, while contrastive learning encourages the model to prefer outputs with higher rewards. These strategies jointly enhance generation quality and promote stronger sarcastic intent in the outputs. Comprehensive evaluations across five metric sets show that ViSP consistently outperforms all baseline models, including Large Language Models, highlighting their limitations in sarcasm generation. Furthermore, analysis of Sarcasm Scores and Factual Incongruity distributions for both M2SaG and ViSP outputs reveals that ViSP achieves higher mean Sarcasm Scores (0.898 vs. 0.770) and Factual Incongruity (0.768 vs. 0.739), demonstrating its ability to produce higher-quality and more contextually sarcastic texts. Our dataset is available at https://github.com/wclapply/ViSP.