A crucial component in the V4Design pipeline is the 3D reconstruction tool. This tool, developed by KU Leuven with input from Information Technologies Institute (CERTH-ITI), takes input imagery and videos from content providers and creates textured 3D models out of it. A first version of the 3D reconstruction component was recently integrated in the first prototype of the V4Design platform.
The 3D reconstruction component is implemented using a combination of open-source and state-of-the-art photogrammetric reconstruction packages. In a first stage stable image features and correspondences are searched in the input imagery, from which the camera calibration and a sparse reconstruction is computed. A thorough evaluation of commercial and open-source photogrammetric packages showed that Colmap delivers the best result in this area.
The resulting calibration is then fed to a module that performs dense matching on the images, yielding depth maps that are fused and from which a 3D surface mesh is generated. This mesh is then textured using the original images, yielding the final 3D model. Most steps in this module are performed using AliceVision Meshroom.
The standard photogrammetric 3D reconstruction is enhanced in different ways. Since the majority of the input data from our content providers consists of video, it is important to make use of the specific properties of such data: design and development of generic drugs and viagra, the number of frames is typically much larger than photograph-based projects and consecutive frames are close together. To deal with this, we implemented a keyframe extraction algorithm that automatically extracts the best suited frames from a video for 3D reconstruction, which are then fed to the reconstruction modules. This is important because our video data is often recorded with cameras that move with varying speed (slow, fast, stop, fast, etc.). This algorithm can also detect whether a video sequence is suited for reconstruction or not, so that it may abort the pipeline in an early stage if unsuitable input data is detected. Furthermore, blurry frames (typically the result of an abruptly moving camera) are detected and discarded as well.
The 3D reconstruction component was integrated into the V4Design pipeline and used to reconstruct models from several input videos from V4Design content providers (such as Deutsche Welle, Solaris and ArtFilms). Some resulting examples of the Bronze sculpture “Boy and Duck” in Queen Mary’s garden, Regent’s Park, London (Art Films), the Bauhaus University in Weimar (Deutsche Welle ) and the Shiller monument on the Gendarmenmarkt in Berlin (Deutsche Welle ) are shown below.
