3D Image Reconstruction from Single 2D Image using Deep Learning
Manoj Kumar Bhargava Pananthula
School of Computer Science and Engineering
Vellore Institute of Technology, Chennai, India
mkbhargava0110@gmail.com
Softya Sebastian
School of Computer Science and Engineering
Vellore Institute of Technology, Chennai, India
softya.sebastian@vit.ac.in
Abstract— Accurate 3D reconstruction from 2D images plays a critical role in various applications including medical imaging, robotics, autonomous navigation, and augmented reality. Traditional reconstruction techniques often require multiple viewpoints or sensor setups, limiting their feasibility in resource-constrained environments. In this work, we propose a deep learning-based monocular 3D reconstruction pipeline that generates high-quality 3D models from a single RGB image. The core of this framework lies in a custom U-Net++ architecture, designed and trained on the NYU Depth V2 dataset for robust depth estimation. This model is evaluated against state-of-the-art alternatives including MiDaS (DPT-Hybrid), Depth Anything V2, and GLPN to assess its performance across accuracy, efficiency, generalization, and visualization quality.
The proposed pipeline performs image preprocessing, depth map prediction, and 3D point cloud generation using Open3D, followed by mesh reconstruction techniques like Poisson Surface Reconstruction. The evaluation metrics include MSE, SSIM, PSNR, and R² Score for depth maps, alongside qualitative analysis of 3D reconstruction quality. Comparative results demonstrate that while GLPN yields the most consistent performance, the Custom U-Net++ model achieves competitive accuracy with significantly improved efficiency and adaptability, making it suitable for real-time or domain-specific deployments.
This research highlights the potential of lightweight, custom-designed architectures for scalable and robust single-view 3D reconstruction. Future directions include multi-view integration, dataset expansion, and enhancing interpretability through uncertainty estimation techniques.
Keywords— Monocular Depth Estimation, 3D Reconstruction, U-Net++, MiDaS, GLPN, Deep Learning, Point Clouds, Open3D.
