Automatic 3D Modeling Process for Predefined Geometrical Categories Based on Convolutional Neural Network and Computer-Vision Analysis of Orthographic Images

Implementing an alternative reality for Metaverse implies modeling optimized digital content to guarantee real-time interaction and high-quality rendering. Even if 3D reconstruction based on 3D scanning techniques provides a good replica of real objects, the output files are challenging to use in this application. On the other hand, manually developed optimized 3D models require much time and effort. This aspect becomes crucial in scenarios including thousands of 3D models with which humans should interact. This paper proposes a method to automate the 3D modeling process of items whose shapes can be classified according to predefined geometrical categories. The dataset for this study relates to products, which present a wide variety of shapes but are attributable to just a few formal archetypes. In the proposed pipeline, metric orthographic images of the object to be digitally reproduced are analyzed by Convolutional Neural Networks (CNN)s. Subsequently, the same images are analyzed with Computer-Vision (CV) algorithms to extrapolate the characteristic dimensions related to the assigned archetypes. The method has been tested on different items, and the results proved the effectiveness of the whole approach in terms of correct archetypes recognition, parameter extraction, and creation of the 3D model, which are comparable with digitized 3D models with high-quality scanning tools but much lighter in model size.