Transformer-based statement level vulnerability detection by cross-modal fine-grained features capture

Software vulnerability detection is crucial for computer systems. The use of fusion information from the source and assembly code can improve the performance of deep learning-based vulnerability detection; however, the bimodal fine-grained alignment information has not been fully utilized in existing methods. Therefore, we propose a cross-modal fine-grained feature capture method based on Transformers for statement-level vulnerability detection. First, we apply a slice generation method based on cross-slicing of bimodal code to obtain bimodal slices. Second, we propose a bimodal-based slice purification method that can effectively shorten the length of source code slices, thereby reducing the impact of vulnerability-unrelated statements on the detection performance. In addition, we replace the information in assembly code that is not conducive to semantic understanding with more easily understandable information, which allows the model to accurately capture the vulnerability features in assembly code. Finally, the fine-grained aligned and purified bimodal code slices are input into the bimodal Transformer model, which can not only capture the vulnerability features within the statements of the bimodal code but also capture the long-term dependencies between statements through context. Compared with existing SOTA methods, the proposed method achieved an average improvement of 6.5% in IOU on the real-world project dataset.