{"month":"04","quality_controlled":"1","has_accepted_license":"1","article_processing_charge":"Yes (via OA deal)","publication_identifier":{"eissn":["1573-1405"],"issn":["0920-5691"]},"user_id":"317138e5-6ab7-11ef-aa6d-ffef3953e345","dataavailabilitystatement":"Due to privacy concerns, we cannot make the dataset used in our project publicly available. However, to demonstrate the effectiveness of our proposed method, we evaluate our approach using publicly available datasets such as H3DS (Ramon et al., 2021): https://github.com/CrisalixSA/h3ds, FFHQ (Karras et al., 2021): https://github.com/NVlabs/ffhq-dataset, and CelebA (Liu et al., 2015): https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html. Using these datasets allows us to evaluate the generalization ability of our proposed method on unseen data. H3DS provides high-quality 3D scans of human faces, FFHQ contains high-resolution facial images, and CelebA is a large-scale dataset of celebrity faces. We use these datasets to evaluate the performance of our proposed method in various scenarios, such as face rotation and relighting, and compare them with state-of-the-art methods.","supplementarymaterial":"no","pmid":1,"publication_status":"published","abstract":[{"text":"Portrait viewpoint and illumination editing is an important problem with several applications in VR/AR, movies, and photography. Comprehensive knowledge of geometry and illumination is critical for obtaining photorealistic results. Current methods are unable to explicitly model in 3D while handling both viewpoint and illumination editing from a single image. In this paper, we propose VoRF, a novel approach that can take even a single portrait image as input and relight human heads under novel illuminations that can be viewed from arbitrary viewpoints. VoRF represents a human head as a continuous volumetric field and learns a prior model of human heads using a coordinate-based MLP with individual latent spaces for identity and illumination. The prior model is learned in an auto-decoder manner over a diverse class of head shapes and appearances, allowing VoRF to generalize to novel test identities from a single input image. Additionally, VoRF has a reflectance MLP that uses the intermediate features of the prior model for rendering One-Light-at-A-Time (OLAT) images under novel views. We synthesize novel illuminations by combining these OLAT images with target environment maps. Qualitative and quantitative evaluations demonstrate the effectiveness of VoRF for relighting and novel view synthesis, even when applied to unseen subjects under uncontrolled illumination. This work is an extension of Rao et al. (VoRF: Volumetric Relightable Faces 2022). We provide extensive evaluation and ablative studies of our model and also provide an application, where any face can be relighted using textual input.","lang":"eng"}],"das_tickbox":"1","oa_version":"Published Version","day":"01","doi":"10.1007/s11263-023-01899-3","volume":132,"publication":"International Journal of Computer Vision","file_date_updated":"2024-07-22T11:07:14Z","title":"A deeper analysis of volumetric relightiable faces","scopus_import":"1","type":"journal_article","status":"public","date_published":"2024-04-01T00:00:00Z","acknowledgement":"Open Access funding enabled and organized by Projekt DEAL.","_id":"14488","file":[{"access_level":"open_access","file_size":9942520,"date_updated":"2024-07-22T11:07:14Z","file_id":"17304","checksum":"5eef1d920f6fe700d7856098000d05f1","file_name":"2024_IJCV_Rao.pdf","creator":"dernst","date_created":"2024-07-22T11:07:14Z","content_type":"application/pdf","relation":"main_file","success":1}],"year":"2024","tmp":{"short":"CC BY (4.0)","image":"/images/cc_by.png","name":"Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)","legal_code_url":"https://creativecommons.org/licenses/by/4.0/legalcode"},"date_created":"2023-11-05T23:00:54Z","publisher":"Springer Nature","page":"1148-1166","external_id":{"isi":["001091935600002"],"pmid":["38549787"]},"citation":{"ama":"Rao P, Mallikarjun BR, Fox G, et al. A deeper analysis of volumetric relightiable faces. International Journal of Computer Vision. 2024;132:1148-1166. doi:10.1007/s11263-023-01899-3","apa":"Rao, P., Mallikarjun, B. R., Fox, G., Weyrich, T., Bickel, B., Pfister, H., … Elgharib, M. (2024). A deeper analysis of volumetric relightiable faces. International Journal of Computer Vision. Springer Nature. https://doi.org/10.1007/s11263-023-01899-3","ieee":"P. Rao et al., “A deeper analysis of volumetric relightiable faces,” International Journal of Computer Vision, vol. 132. Springer Nature, pp. 1148–1166, 2024.","ista":"Rao P, Mallikarjun BR, Fox G, Weyrich T, Bickel B, Pfister H, Matusik W, Zhan F, Tewari A, Theobalt C, Elgharib M. 2024. A deeper analysis of volumetric relightiable faces. International Journal of Computer Vision. 132, 1148–1166.","mla":"Rao, Pramod, et al. “A Deeper Analysis of Volumetric Relightiable Faces.” International Journal of Computer Vision, vol. 132, Springer Nature, 2024, pp. 1148–66, doi:10.1007/s11263-023-01899-3.","short":"P. Rao, B.R. Mallikarjun, G. Fox, T. Weyrich, B. Bickel, H. Pfister, W. Matusik, F. Zhan, A. Tewari, C. Theobalt, M. Elgharib, International Journal of Computer Vision 132 (2024) 1148–1166.","chicago":"Rao, Pramod, B. R. Mallikarjun, Gereon Fox, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, et al. “A Deeper Analysis of Volumetric Relightiable Faces.” International Journal of Computer Vision. Springer Nature, 2024. https://doi.org/10.1007/s11263-023-01899-3."},"fulldoi":"https://doi.org/10.1007/s11263-023-01899-3","article_type":"original","isi":1,"intvolume":" 132","author":[{"full_name":"Rao, Pramod","first_name":"Pramod","last_name":"Rao"},{"last_name":"Mallikarjun","first_name":"B. R.","full_name":"Mallikarjun, B. R."},{"last_name":"Fox","full_name":"Fox, Gereon","first_name":"Gereon"},{"full_name":"Weyrich, Tim","first_name":"Tim","last_name":"Weyrich"},{"last_name":"Bickel","id":"49876194-F248-11E8-B48F-1D18A9856A87","orcid":"0000-0001-6511-9385","first_name":"Bernd","full_name":"Bickel, Bernd"},{"last_name":"Pfister","first_name":"Hanspeter","full_name":"Pfister, Hanspeter"},{"last_name":"Matusik","full_name":"Matusik, Wojciech","first_name":"Wojciech"},{"first_name":"Fangneng","full_name":"Zhan, Fangneng","last_name":"Zhan"},{"full_name":"Tewari, Ayush","first_name":"Ayush","last_name":"Tewari"},{"full_name":"Theobalt, Christian","first_name":"Christian","last_name":"Theobalt"},{"last_name":"Elgharib","full_name":"Elgharib, Mohamed","first_name":"Mohamed"}],"ddc":["000"],"oa":1,"license":"https://creativecommons.org/licenses/by/4.0/","researchdata_availability":"no (restricted access)","date_updated":"2026-09-24T10:59:50Z","department":[{"_id":"BeBi"}],"language":[{"iso":"eng"}]}