Controlling Material Properties in Images

Paper Title:

Alchemist: Parametric Control of Material Properties with Diffusion Models

Published on:

5 December 2023

Primary Category:

Computer Vision and Pattern Recognition

Paper Authors:

Prafull Sharma,

Varun Jampani,

Yuanzhen Li,

Xuhui Jia,

Dmitry Lagun,

Fredo Durand,

William T. Freeman,

Mark Matthews

Bullets

Key Details

•

Proposes method to control material properties in images

•

Renders synthetic dataset with explicit material annotations

•

Fine-tunes text-to-image model on this dataset

•

Edits roughness, metallic, albedo and transparency

•

Works directly in pixel space without graphics pipelines

•

Can also edit materials in neural radiance fields

Explore the topics in this paper

image synthesis

material editing

material properties

neural radiance fields

text-to-image models

AI generated summary

Controlling Material Properties in Images

This paper proposes a method to precisely control key material properties like roughness, metallic, albedo and transparency in real images. It uses a text-to-image model fine-tuned on a synthetic dataset with explicit material annotations. This allows editing materials in images while retaining all other attributes, without needing to estimate geometry, lighting etc. They show it can also edit neural radiance fields.