Burden-Reduced Cleft Lip and Palate Care and Healing (BRCCH)

Cleft lip and palate is the most frequent craniofacial malformation in newborns, without effective preventive measures. The use of intra-oral orthopedic plates reduces the cleft size, facilitating surgical treatment.  This project aims at an automatic, image-based design (e.g., using smartphone videos) of personalized oral plates that are fabricated using 3D printers. Ultimately, the goal is to facilitate the use of plate therapy in low-income countries.  This project is funded by the Botnar Research Center for Child Health (BRCCH) and implemented in collaboration with the Computer Graphics Lab (CGL) at ETH Zürich and the team of Dr. Andreas Müller from the University Hospital in Basel.  [Project page]

C. Otto, J. Naruniec,  L. Helminger, T. Etterlin, G. Mignone, P. Chandran, G. Zoss, C. Schroers, M. Gross, P. Gotardo, D. Bradley, R. Weber

Learning Dynamic 3D Geometry and Texture for Video Face Swapping

Pacific Graphics 2022

We approach face swapping as learning simultaneous facial autoencoders for the source and target identities, using a shared encoder network with identity-specific decoders. Our decoders first lift the latent code into a 3D representation, before using a differentiable renderer, thus allowing for artistic control over the result. Training does not require 3D supervision, leading to better results than when using off-the-shelf monocular 3D face reconstruction. [Project page]

Daoye Wang, Prashanth Chandran, Gaspard Zoss, Derek Bradley, Paulo Gotardo

MoRF: Morphable Radiance Fields for Multiview Neural Head Modeling

SIGGRAPH 2022

We present MoRF, morphable radiance fields that extend NeRFs into generative models for synthesizing photorealistic human heads with controllable and fully disentangled identity and 3D pose. MoRF allows for applications such as synthesizing new photorealistic subjects or quickly fitting a NeRF to one or more full-head portrait images.  [Project page]

Yingyan Xu, Jérémy Riviere, Gaspard Zoss, Prashanth Chandran, Derek Bradley, Paulo Gotardo,

Improved Lighting Models for Facial Appearance Capture

Eurographics 2022, short paper

We compare the results obtained with a state-of-the-art appearance capture method [RGB∗20], with and without our proposed improvements to the lighting model. [Project page]

Prashanth Chandran, Sebastian Winberg, Gaspard Zoss, Jérémy Riviere, Markus Gross, Paulo Gotardo, Derek Bradley

Rendering with Style: Combining Traditional and Neural Approaches for High-Quality Face Rendering

SIGGRAPH Asia 2021

We propose to combine incomplete, high-quality renderings showing only facial skin with recent methods for neural rendering of faces, in order to automatically and seamlessly create photo-realistic full-head portrait renders from captured data without the need for artist intervention. [Project page]

Jérémy Riviere, Paulo Gotardo, Derek Bradley, Abhijeet Ghosh, Thabo Beeler

Single-Shot High-Quality Facial Geometry and Skin Appearance Capture

SIGGRAPH 2020

We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. [Project page]

Zdravko Velinov, Marios Papas, Derek Bradley, Paulo Gotardo, Parsa Mirdehghan, Steve Marschner, Jan Novak, Thabo Beeler

Appearance Capture and Modeling of Human Teeth

SIGGRAPH Asia 2018

We present a system specifically designed for capturing the optical properties of live human teeth such that they can be realistically re-rendered in computer graphics. [Project page]

Paulo Gotardo, Tomas Simon, Yaser Sheikh, Iain Matthews

Photogeometric Scene Flow for High-Detail Dynamic 3D Reconstruction

ICCV 2015

This paper proposes photogeometric scene flow (PGSF) for high-quality dynamic 3D reconstruction. Results are obtained as the coupled solution of multiview stereo, photometric stereo, and optical flow (with relighting). [Project page]

Paulo Gotardo, Onur Hamsici, Aleix Martinez

Learning Spatially-Smooth Mappings in Non-Rigid Structure from Motion (ECCV 2011)

Kernel Non-Rigid Structure from Motion (ICCV 2011)

Non-Rigid Structure from Motion with Complementary Rank-3 Spaces (CVPR 2011)

Computing Smooth Time-Trajectories for Camera and Deformable Shape in Structure from Motion with Occlusion (IEEE Trans. PAMI, 33(10), 2011)

While I was a PhD student and then postdoc at The Ohio State University, I developed state-of-the-art models and algorithms for matrix factorization and non-rigid structure from motion (NR-SfM), which were published in main computer vision venues and subsequently achieved 2nd, 3rd, and 4th places in the first NR-SfM challenge at CVPR 2017. More info and source codes are found on my old OSU home page, here: [Project page]

Paulo Gotardo, Kim L. Boyer, Joel Saltz, Subha V. Raman

A New Deformable Model for Boundary Tracking in Cardiac MRI and Its Application to the Detection of Intra-ventricular Dyssynchrony

CVPR 2006

Intra-ventricular dyssynchrony (IVD) in the left ventricle (LV) is caused by the asynchronous activation of the LV walls. Guidelines for resynchronization therapy rely on measures that do not reliably predict successful patient response to treatment, in part due to poor characterization of IVD. We present a two-class statistical pattern recognition approach for the detection of IVD in the LV from routinely acquired MRI sequences depicting complete cardiac cycles.