Extracting Materials from Photos

Paper Title:

Material Palette: Extraction of Materials from a Single Image

Published on:

28 November 2023

Primary Category:

Computer Vision and Pattern Recognition

Paper Authors:

Ivan Lopes,

Fabio Pizzati,

Raoul de Charette

Bullets

Key Details

•

Proposes novel task of material extraction from photos

•

Uses text-to-image model to generate tileable material textures

•

Decomposes textures into albedo, normal, roughness maps

•

Validation on synthetic and real-world material datasets

•

Showcases material editing of 3D scenes

Explore the topics in this paper

inverse rendering

material editing

material synthesis

neural rendering

texture synthesis

AI generated summary

Extracting Materials from Photos

This paper proposes a two-step method to extract realistic 3D materials from regions in a single photo, without needing multi-view captures or 3D geometry. First, a text-to-image model generates tileable texture images resembling the region's material. Then a domain-adapted neural network decomposes textures into reusable albedo, normal, roughness maps. Experiments validate the quality of extracted materials and showcase editing of 3D scenes.