TY - JOUR
T1 - DPPS
T2 - A deep-learning based point-light photometric stereo method for 3D reconstruction of metallic surfaces
AU - Yang, Ru
AU - Wang, Yaoke
AU - Liao, Shuheng
AU - Guo, Ping
N1 - Funding Information:
This research was supported by the start-up fund from McCormick School of Engineering, Northwestern University, USA ; and the National Science Foundation, USA under Grant number EEC-2133630 and CNS-2229170 . The authors would like to thank Dohyun Leem for providing the metal forming part and Guangze Li for providing the 3D camera.
Publisher Copyright:
© 2023 Elsevier Ltd
PY - 2023/3/31
Y1 - 2023/3/31
N2 - Three-dimensional (3D) measurement provides essential geometric information for quality control and process monitoring in many manufacturing applications. Photometric stereo is one of the potential solutions for in-process metrology and active geometry compensation, which takes multiple images of an object under different illuminations as inputs and recovers its surface normal map based on a reflectance model. Deep learning approaches have shown their potential in solving the highly nonlinear problem for photometric stereo, but the main challenge preventing their practical application in process metrology lies in the difficulties in the generation of a comprehensive dataset for training the deep learning model. This paper presents a new Deep-learning based Point-light Photometric Stereo method, DPPS, which utilizes a multi-channel deep convolutional neural network (CNN) to achieve end-to-end prediction for both the surface normal and height maps in a semi-calibrated fashion. The key contribution is a new dataset generation method combining both physics-based and data-driven approaches, which minimizes the training cost and enables DPPS to handle reflective metal surfaces with unknown surface roughness. Even trained only with fully synthetic and high-fidelity dataset, our DPPS surpasses the state-of-the-art with an accuracy better than 0.15 cm over a 10 cm × 10 cm area and its real-life experimental results are on par with commercial 3D scanners. The demonstrated results provide guidance on improving the generalizability and robustness of deep-learning based computer vision metrology with minimized training cost as well as show the potential for in-process 3D metrology in advanced manufacturing processes.
AB - Three-dimensional (3D) measurement provides essential geometric information for quality control and process monitoring in many manufacturing applications. Photometric stereo is one of the potential solutions for in-process metrology and active geometry compensation, which takes multiple images of an object under different illuminations as inputs and recovers its surface normal map based on a reflectance model. Deep learning approaches have shown their potential in solving the highly nonlinear problem for photometric stereo, but the main challenge preventing their practical application in process metrology lies in the difficulties in the generation of a comprehensive dataset for training the deep learning model. This paper presents a new Deep-learning based Point-light Photometric Stereo method, DPPS, which utilizes a multi-channel deep convolutional neural network (CNN) to achieve end-to-end prediction for both the surface normal and height maps in a semi-calibrated fashion. The key contribution is a new dataset generation method combining both physics-based and data-driven approaches, which minimizes the training cost and enables DPPS to handle reflective metal surfaces with unknown surface roughness. Even trained only with fully synthetic and high-fidelity dataset, our DPPS surpasses the state-of-the-art with an accuracy better than 0.15 cm over a 10 cm × 10 cm area and its real-life experimental results are on par with commercial 3D scanners. The demonstrated results provide guidance on improving the generalizability and robustness of deep-learning based computer vision metrology with minimized training cost as well as show the potential for in-process 3D metrology in advanced manufacturing processes.
KW - 3D reconstruction
KW - Convolutional neural network
KW - Deep learning
KW - Photometric stereo
KW - Point light
UR - http://www.scopus.com/inward/record.url?scp=85147254176&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85147254176&partnerID=8YFLogxK
U2 - 10.1016/j.measurement.2023.112543
DO - 10.1016/j.measurement.2023.112543
M3 - Article
AN - SCOPUS:85147254176
SN - 0263-2241
VL - 210
JO - Measurement: Journal of the International Measurement Confederation
JF - Measurement: Journal of the International Measurement Confederation
M1 - 112543
ER -