Neural 3D reconstruction is a growing field in computer vision focused on generating 3D representations from 2D images or videos. Learning algorithms, such as Neural Radiance Fields (NeRFs), play a significant role in this approach. NeRFs are neural networks that model the 3D shape and color of objects in a continuous 3D space. Through training on multiple 2D images, NeRFs can predict radiance at any 3D point, enabling the creation of realistic and detailed 3D models.
Currently there are limited applications of NeRFs with real-time scanning. There are recent works such as Nerfcapture and Nerf_Bridge that attempt to address it. These scanning apps include their own set of limitations. We want to evaluate these methods, build up on their implementations and eliminate their limitations to develop a scanning system that performs the following steps in incremental/online fashion:
Capture images -> camera calibration -> stream data to NeRF training pipeline -> monitor performance
For more information about our 3D reconstruction research, please refer to
https://www.iis.fraunhofer.de/en/profil/zukunftsinitiativen/artificial-intelligence/dsai.html
Piqued your interest? Then have a look at our offer!