Gallery of stylization results. Original renderings were posterized using the palettes extracted by our visibility-weighted method and recolored by changing the marked palette colors (left-to-right).
Abstract
Volumetric radiance fields have recently gained significant attention as promising representations of photorealistic scene reconstruction. However, the non-photorealistic rendering of such a representation has barely been explored. In this study, we investigate the artistic posterization of the volumetric radiance fields. We extend the recent palette-based image-editing framework, which naturally introduces intuitive color manipulation of the posterized results, into the radiance field. Our major challenge is applying stylization effects coherently across different views. Based on the observation that computing a palette frame-by-frame can produce flickering, we propose pre-computing a single palette from the volumetric radiance field covering its entire visible color. We present a method based on volumetric visibility to sample visible colors from the radiance field while avoiding occluded and noisy regions. We demonstrate our workflow by applying it to pre-trained volumetric radiance fields with various stylization effects. We also show that our approach can produce more coherent and robust stylization effects than baseline methods that compute a palette on each rendered view.
Resources
Paper / Supplemental video / Code / Presentation slides / Presentation video
Citation
@article{tojo2022posternerf,
title = {Recolorable Posterization of Volumetric Radiance Fields Using Visibility-Weighted Palette Extraction},
author = {Tojo, Kenji and Umetani, Nobuyuki},
journal = {Computer Graphics Forum},
number = {4},
pages = {149-160},
volume = {41},
year = {2022}
}