Abstract
Advancements in neural signed distance felds (SDFs) have enabled modeling 3D surface geometry from a set of 2D images of real-world scenes. Baking neural SDFs can extract explicit mesh with appearance baked into texture maps as neural features. The baked meshes still have a large memory footprint and require a powerful GPU for real-time rendering. Neural optimization of such large meshes with differentiable rendering pose signifcant challenges. We propose a method to produce optimized meshes for large unbounded scenes with low triangle budget and high fdelity of geometry and appearance. We achieve this by combining advancements in baking neural SDFs with classical mesh simplifcation techniques and proposing a joint appearance-geometry refnement step. The visual quality is comparable to or better than state-of-the-art neural meshing and baking methods with high geometric accuracy despite signifcant reduction in triangle count, making the produced meshes effcient for storage, transmission, and rendering on mobile hardware. We validate the effectiveness of the proposed method on large unbounded scenes from mip-NeRF 360, Tanks & Temples, and Deep Blending datasets, achieving at-par rendering quality with 73× reduced triangles and 11× reduction in memory footprint.
- Task: Signed Distance Field Rendering Optimization
- Problem Definition: The baked meshes still have a large memory footprint and require a powerful GPU for real-time rendering
- Approach: propose a method to produce optimized meshes for large unbounded scenes with low triangle budget and high fdelity of geometry and appearance