Shape-from-shifting: Uncalibrated photometric stereo with a mobile device

Surface shape scanning techniques, such as laser scanning and photometric stereo, are widespread analytical tools used in the field of cultural heritage. Compared to regular 2D RGB photos, 3D surface scans provide higher fidelity of an object's surface shape which assist conservators, art historians, and archaeologists in understanding how these artworks and artifacts are made and to digitally document them for purposes of conservation. However, current state-of-the-art 3D surface scanning tools used in art conservation are often expensive and bulky-such as light dome structures that are often over 1 m in diameter. In this paper, we introduce mobile shape-from-shifting (SfS): A simple, low-cost and streamlined photometric stereo framework for scanning planar surfaces with a consumer mobile device coupled to a low-cost add-on component. Our free-form mobile SfS framework relaxes the rigorous hardware and other complex requirements inherent to conventional 3D scanning tools. This is achieved by taking a sequence of photos with the on-board camera and flash of a mobile device. The sequence of captures are used to reconstruct high quality normal maps using nearlight photometric stereo algorithms, which are of comparable quality to conventional photometric stereo. We demonstrate 3D surface reconstructions with SfS on different materials and scales. Moreover, the mobile SfS technique can be used 'in the wild' so that 3D scans may be performed in their natural environment, eliminating the need for transport to a laboratory setting. With the elegant design and low cost, we believe our Mobile SfS can greatly benefit the conservation community by providing a userfriendly and cost-effective solution for 3D surface scanning.