Nail Ibrahimli

Nail Ibrahimli

Doctoral Student, 3D Vision

Talks and presentations

Recent development in learning-based 3D Reconstruction

November 17, 2022

Talk, 3D Urban Understanding Lab,

Since 2015, there has been ongoing research on stereo learning. Siamese networks were initially employed to densely match the patches. Cost volume regularization-based stereo techniques have become more common since 2017. Reconstructing more complete 3D models was made possible by learning-based multi view stereo.Approaches based on differentiable rendering and neural rendering have recently gained in popularity. Utilizing positional encoding and volumetric rendering, it was feasible to reconstruct surfaces with non-Lambertian surfaces in addition to synthesizing novel viewpoints.We will go through the both paradigms’ most recent developments (stereo-based and neural rendering-based) during this talk.

Volumetric differentiable rendering

July 11, 2022

Talk, Scene Understanding and Modeling Webinar Series, SUM Webinar

Lately, learning-based 3D reconstruction methods have shown impressive results. There is a line of research in the last couple of years that unlike most traditional and other learning-based methods does not require 3D supervision which is often hard to obtain for real-world datasets. Recently, several works have proposed differentiable rendering techniques to train reconstruction models from RGB images. The approaches that are restricted to voxel- and mesh-based representations, suffer from discretization or low resolution. In this webinar, we will talk about a differentiable rendering formulation for implicit shape and texture representations. Implicit representations have recently gained popularity as they represent shape and texture continuously. I will present the work DVR (Niemeyer et al. 2020) which shows that depth gradients can be derived analytically using the concept of implicit differentiation. This allows neural networks to learn implicit shape and texture representations directly from RGB images. DVR can be used for multi-view 3D reconstruction, directly resulting in watertight meshes.

Join Zoom Meeting via this link

Paper presentation: Manhattan SDF

June 16, 2022

Talk, 3D Urban Understanding Lab, CVPR22 Paper

Manhattan SDF addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Many previous works have shown impressive reconstruction results on textured objects, but they still have difficulty in handling low-textured planar regions, which are common in indoor scenes. An approach to solving this issue is to incorporate planar constraints into the depth map estimation in multi-view stereo-based methods, but the per-view plane estimation and depth optimization lack both efficiency and multi-view consistency. In this work, authors show that the planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods. Specifically, the work uses an MLP network to represent the signed distance function as the scene geometry. Based on the Manhattan-world assumption, planar constraints are employed to regularize the geometry in floor and wall regions predicted by a 2D semantic segmentation network. To resolve the inaccurate segmentation, Manhattan SDF encodes the semantics of 3D points with another MLP and design a novel loss that jointly optimizes the scene geometry and semantics in 3D space. Experiments on ScanNet and 7-Scenes datasets show that the proposed method outperforms previous methods by a large margin on 3D reconstruction quality.

Project page can be found in this link