A method suitable for tunnel inspection robots to diagnose multi-defects of structures using the extraction of hybrid visual features

Robots using visual inspection represent a new tool to diagnose tunnel defects. However, due to the combination of several factors, such as insufficient illumination, vibration of rail joints and mechanical structures, the quality of tunnel images is inevitably distorted, which affects the accuracy of defect diagnosis. In addition, the existing defect detection methods driven by visual recognition models, primarily structured around convolutional neural networks (CNNs), always fail to determine the relation between the macro-scale and micro-scale visual information, limiting the accuracy of detecting multi-defects in tunnels. To address the above-mentioned issues, a method suitable for tunnel inspection robots to diagnose multi-defects of structures using the extraction of hybrid visual features is proposed. First, a network architecture of the attention mechanism was optimized by combing a convolutional neural network and the vision transformer (ViT). Second, with the generated attention mechanism, a method for extracting the hybrid visual features was investigated by considering the multi-level distribution characteristics of visual features obtained by the inspection robots. Third, multi-scale features of images were generated by utilizing the proposed visual feature extraction network, and then, an inspection robot multi-defect detection method for tunnels was established. Finally, the effectiveness of the proposed method was validated using the visual images collected from an actual tunnel by inspection robots, which achieve a maximum accuracy of over 90% for identifying the three typical apparent defects in tunnels. When the visual image data of tunnels under the state without structural disease or damage are lacking, preliminary optimization of the attention mechanism–optimized visual feature combination extraction network cannot be performed, making it difficult to ensure the identification accuracy of the proposed method.