Query-based Collaborative Multimodal Token Pruning for Audio-Visual Question Answering

Audio-Visual Question Answering (AVQA) requires complex reasoning across auditory and visual modalities. While recent advancements leverage sophisticated spatio-temporal representations, cross-modal fusion, and question-aware mechanisms to boost performance, they frequently neglect the computational burden and significant information redundancy inherent in processing long, dense video sequences. This inefficiency stems from frame-by-frame processing where critical information is often sparse. To bridge this efficiency gap, this paper introduces a novel Query-based Collaborative Multimodal Token Pruning strategy. Our method exploits the coupling relationship across modalities between layers to collaboratively assesses multimodal token importance required for the current state and update the query accordingly by the proposed Adaptive Multimodal Token Pruning. This query-based collaboration enables highly efficient, focused reasoning. Experiments on the MUSIC-AVQA dataset demonstrate that our method prunes up to 80% of multimodal tokens without accuracy degradation, validating our query-based collaborative multimodal token pruning as a highly effective strategy for building more efficient and robust AVQA models.