Integration of Point Clouds from 360° Videos and Deep Learning Techniques for Rapid Documentation and Classification in Historical City Centers

Digital metric documentation of historical city centers is challenging because of the complexity of the buildings and monuments, which feature different geometries, construction technologies, and materials. We propose a solution for rapid documentation and classification of such complex spaces using 360° video cameras, which can capture the entire scene and can be pointed in any direction, making data acquisition rapid and straightforward. The high framerate during image acquisition allows users to capture overlapping images that can be used for photogrammetric applications. This paper aims to quickly capture 360° videos with low-cost cameras and then generate dense point clouds using the photogrammetric/structure from motion pipeline for 3D modeling. Point cloud classification is the prerequisite for such applications. Numerous deep learning methods (DL) have been developed to classify point clouds due to the expansion of artificial intelligence (AI) capabilities. We aim to pave the way toward utilizing the convolutional neural network (CNN) to classify point clouds generated by 360° videos of historic cities. A preliminary case study in a historic city center demonstrates that our method achieves promising results in the generation and classification of point clouds, with an overall classification accuracy of 96% using the following categories: ground, buildings, poles, bollards, cars, and natural.