Vision-based multi-level synthetical evaluation of seismic damage for RC structural components: a multi-task learning approach

Recent studies for computer vision and deep learning-based, post-earthquake inspections on RC structures mainly perform well for specific tasks, while the trained models must be fine-tuned and re-trained when facing new tasks and datasets, which is inevitably time-consuming. This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components, three-type seismic damage, and four-type deterioration states. The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head, task-specific recognition subnetwork. The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures. The multi-head, task-specific recognition subnetwork consists of three individual self-attention pipelines, each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task. A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one. Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity. The results show that the proposed method can simultaneously recognize different structural components, seismic damage, and deterioration states, and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.