Inverse Rendering - Paper Copilot

DANI-Net: Uncalibrated Photometric Stereo by Differentiable ShadowHandling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering

Zongrui Li, Qian Zheng, Boxin Shi, Gang Pan, Xudong Jiang

Nanyang Technological University; Zhejiang University; Peking University

Portals

Abstract

Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by the unknown light. Although the ambiguity is alleviated on non-Lambertian objects, the problem is still difficult to solve for more general objects with complex shapes introducing irregular shadows and general materials with complex reflectance like anisotropic reflectance. To exploit cues from shadow and reflectance to solve UPS and improve performance on general materials, we propose DANI-Net, an inverse rendering framework with differentiable shadow handling and anisotropic reflectance modeling. Unlike most previous methods that use non-differentiable shadow maps and assume isotropic material, our network benefits from cues of shadow and anisotropic reflectance through two differentiable paths. Experiments on multiple real-world datasets demonstrate our superior and robust performance.

Related Works

Unsupervised calibrated photometric stereo; Uncalibrated photometric stereo; Shadow handling in photometric stereo; Neural reflectance representation in 3D vision

Comparisons

LL22, SCPS-NIR

Look-Ahead Training with Learned Reflectance Loss for Single-Image SVBRDF Estimation

Xilong Zhou, Nima Kalantari

Texas A&M University

Portals

Abstract

In this paper, we propose a novel optimization-based method to estimate the reflectance properties of a near planar surface from a single input image. Specifically, we perform test-time optimization by directly updating the parameters of a neural network to minimize the test error. Since single image SVBRDF estimation is a highly ill-posed problem, such an optimization is prone to overfitting. Our main contribution is to address this problem by introducing a training mechanism that takes the test-time optimization into account. Specifically, we train our network by minimizing the training loss after one or more gradient updates with the test loss. By training the network in this manner, we ensure that the network does not overfit to the input image during the test-time optimization process. Additionally, we propose a learned reflectance loss to augment the typically used rendering loss during the test-time optimization. We do so by using an auxiliary network that estimates pseudo ground truth reflectance parameters and train it in combination with the main network. Our approach is able to converge with a small number of iterations of the test-time optimization and produces better results compared to the state-of-the-art methods.

Related Works

Direct-Estimation Methods; Optimization-Based Methods; Meta-Learning

BakedSDF: Meshing Neural SDFs for Real-time View Synthesis

Error: Cannot create object

Category: Inverse Rendering

Portals

Abstract

Related Works

Comparisons

Portals

Abstract

Related Works