Measurement-based modeling and fabrication of deformable materials for human faces

This thesis investigates the combination of data-driven and physically based techniques for acquiring, modeling, and animating deformable materials, with a special focus on human faces. Furthermore, based on these techniques, we introduce a data-driven process for designing and fabricating materials with desired deformation behavior. Realistic simulation behavior, surface details, and appearance are still demanding tasks. Neither pure data-driven, pure procedural, nor pure physical methods are best suited for accurate synthesis of facial motion and details (both for appearance and geometry), due to the difficulties in model design, parameter estimation, and desired controllability for animators. Capturing of a small but representative amount of real data, and then synthesizing diverse on-demand examples with physically-based models and real data as input benefits from both sides: Highly realistic model behavior due to real-world data and controllability due to physically-based models. To model the face and its behavior, hybrid physically-based and data-driven approaches are elaborated. We investigate surface-based representations as well as a solid representation based on FEM. To achieve realistic behavior, we propose to build light-weighted data capture devices to acquire real-world data to estimate model parameters and to employ concepts from data-driven modeling techniques and machine learning. The resulting models support simple acquisition systems, offer techniques to process and extract model parameters from real-world data, provide a compact representation of the facial geometry and its motion, and allow intuitive editing. We demonstrate applications such as capture of facial geometry and motion and real-time animation and transfer of facial details, and show that our soft tissue model can react to external forces and produce realistic deformations beyond facial expressions. Based on this model, we furthermore introduce a data-driven process for designing and fabricating materials with desired deformation behavior. The process starts with measuring deformation properties of base materials. Each material is represented as a non-linear stress-strain relationship in a finite-element model. For material design and fabrication, we introduce an optimization process that finds the best combination of base materials that meets a user’s criteria specified by example deformations. Our algorithm employs a number of strategies to prune poor solutions from the combinatorial search space. We finally demonstrate the complete process by designing and fabricating objects with complex heterogeneous materials using modern multi-material 3D printers.