TY - JOUR
T1 - Attention-guided analysis of infrastructure damage with semi-supervised deep learning
AU - Karaaslan, Enes
AU - Bagci, Ulas
AU - Catbas, F. Necati
N1 - Funding Information:
The project presented in this paper was partially funded by NCHRP- IDEA Project 222 (IDEA Program Transportation Research Board of The National Academies) and Nexco-West (West Nippon Expressway Authority). The authors are grateful for their support. The findings and results presented in this paper are the views of the authors, not necessarily views of the sponsoring agencies.
Publisher Copyright:
© 2021
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2021/5
Y1 - 2021/5
N2 - Routine visual inspection is essential to maintain adequate safety and serviceability of civil infrastructures. Computer vision and machine learning based software techniques are becoming recognized methods that can potentially help the inspectors analyze the physical and functional condition of infrastructures from images and/or videos of the region of interest. More recently, deep learning approaches have been shown robust in identifying damages; yet these methods require precisely labeled large amount of training data for high accuracy complementary to visual assessment of inspectors. Especially in image segmentation operations, in which damages are subtracted from the image background for further analysis, there is a strong need to localize the damaged region prior to segmentation operation. However, available segmentation methods mostly focus on the latter step (i.e., delineation), and mis-localization of damaged regions causes accuracy drops. Inspired by the superiority of human cognitive system, where recognizing objects is simpler and more efficient than machine learning algorithms, which are superior to human in local tasks, this paper describes a novel method to dramatically improve the accuracy of the damage quantification (detection + segmentation) using an attention-guided technique. In the proposed method, a fast object detection model, Single Shot Detector (SSD) trained on VGG-16 base classifier architecture, performs a real-time crack and spall detection while working interactively with the human inspector to ensure recognition of the region of interest is well-performed. Upon the inspector's verification, happening in real-time, the detected damage region is used for damage segmentation for further analysis. This initial region of interest selection drastically lowers the computational cost, required amount of training data and reduces number of outliers. For optimal performance, a modified version of SegNet architecture was used for damage segmentation. Based on various performance criteria, the proposed attention-guided infrastructure damage analysis technique provides 30% more precision with a very minor sacrifice in computational speed compared to analysis without using attention guide.
AB - Routine visual inspection is essential to maintain adequate safety and serviceability of civil infrastructures. Computer vision and machine learning based software techniques are becoming recognized methods that can potentially help the inspectors analyze the physical and functional condition of infrastructures from images and/or videos of the region of interest. More recently, deep learning approaches have been shown robust in identifying damages; yet these methods require precisely labeled large amount of training data for high accuracy complementary to visual assessment of inspectors. Especially in image segmentation operations, in which damages are subtracted from the image background for further analysis, there is a strong need to localize the damaged region prior to segmentation operation. However, available segmentation methods mostly focus on the latter step (i.e., delineation), and mis-localization of damaged regions causes accuracy drops. Inspired by the superiority of human cognitive system, where recognizing objects is simpler and more efficient than machine learning algorithms, which are superior to human in local tasks, this paper describes a novel method to dramatically improve the accuracy of the damage quantification (detection + segmentation) using an attention-guided technique. In the proposed method, a fast object detection model, Single Shot Detector (SSD) trained on VGG-16 base classifier architecture, performs a real-time crack and spall detection while working interactively with the human inspector to ensure recognition of the region of interest is well-performed. Upon the inspector's verification, happening in real-time, the detected damage region is used for damage segmentation for further analysis. This initial region of interest selection drastically lowers the computational cost, required amount of training data and reduces number of outliers. For optimal performance, a modified version of SegNet architecture was used for damage segmentation. Based on various performance criteria, the proposed attention-guided infrastructure damage analysis technique provides 30% more precision with a very minor sacrifice in computational speed compared to analysis without using attention guide.
KW - Attention guided segmentation
KW - Concrete inspection
KW - Crack and spall detection
KW - Damage segmentation
KW - Infrastructure assessment
KW - Semi-supervised learning
UR - http://www.scopus.com/inward/record.url?scp=85101541960&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85101541960&partnerID=8YFLogxK
U2 - 10.1016/j.autcon.2021.103634
DO - 10.1016/j.autcon.2021.103634
M3 - Article
AN - SCOPUS:85101541960
VL - 125
JO - Automation in Construction
JF - Automation in Construction
SN - 0926-5805
M1 - 103634
ER -