Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Prior. (arXiv:2306.11739v1 [cs.CV])

3D object reconstruction is important for semantic scene understanding. It is
challenging to reconstruct detailed 3D shapes from monocular images directly
due to a lack of depth information, occlusion and noise. Most current methods
generate deterministic object models without any awareness of the uncertainty
of the reconstruction. We tackle this problem by leveraging a neural object
representation which learns an object shape distribution from large dataset of
3d object models and maps it into a latent space. We propose a method to model
uncertainty as part of the representation and define an uncertainty-aware
encoder which generates latent codes with uncertainty directly from individual
input images. Further, we propose a method to propagate the uncertainty in the
latent code to SDF values and generate a 3d object mesh with local uncertainty
for each mesh component. Finally, we propose an incremental fusion method under
a Bayesian framework to fuse the latent codes from multi-view observations. We
evaluate the system in both synthetic and real datasets to demonstrate the
effectiveness of uncertainty-based fusion to improve 3D object reconstruction
accuracy.