授课讲者：Shuang Zhao (赵爽), University of California, Irvine
课程摘要：Physics-based rendering algorithms generate photorealistic images by simulating the flow of light through a detailed mathematical representation of a virtual scene. In contrast, physics-based differentiable rendering algorithms compute derivatives of images exhibiting complex light transport effects (e.g., soft shadows, interreflections, multiple scattering, and caustics) with respect to arbitrary scene parameters (e.g., camera pose, object geometry, spatially varying material properties).
This new level of generality has made physics-based differentiable rendering a key ingredient in inferential pipelines for computer vision and computer graphics applications. For example, differentiable renderers can be used to solve previously intractable analysis-by-synthesis problems, that is, the search of scene configurations optimizing user-specified objective functions, using gradient-based methods (as illustrated above). Additionally, differentiable rendering can be incorporated into probabilistic inference and machine learning architecture. For example, differentiable renderers allow backpropagating through “image losses” that capture complex light transport effects, providing a robust physics-aware regularization when training neural networks. Likewise, differentiable renderers can be combined with generative models to synthesize photorealistic images.
Compared to its “ordinary” counterpart, physics-based differentiable rendering introduces unique theoretical and engineering challenges. For instance, boundaries of objects introduce troublesome discontinuities during the computation of shadows and interreflections that lead to incorrect gradients if precautions are not taken.
Thankfully, recent advances in physics-based differentiable rendering theory have enabled the development of unbiased Monte Carlo algorithms that overcome the above challenges, making it possible to estimate derivatives of radiometric measurements with respect to arbitrary scene parameters in a computationally efficient way. In this talk, I will provide a high-level overview on physics-based differentiable rendering, highlighting relevant theory, algorithms, implementations, as well as current and future applications in computer vision and related areas.
讲者简介：I am an Assistant Professor of Computer Science at the University of California, Irvine (UCI) and co-direct UCI's Interactive Graphics & Visualization Lab (iGravi). Before joining UCI, I was a postdoctoral associate at MIT. I received my Ph.D. in computer science from Cornell University in 2014.
My group works mainly in physics-based computer graphics and scientific computing with a focus on modeling and simulating how light interacts with complex materials (e.g., cloth, animal fur, and human skin). Additionally, we aim to develop efficient solutions to the inverse problems centered around the inference of geometric or material properties from physical or simulated measurements.
讲者简介：翟晓雅, 中国科学技术大学博士后研究员, 主要研究方向为计算机辅助几何设计、微结构设计、仿生骨结构设计等。研究工作主要发表在Computer-Aided Design、Structural and Multidisciplinary Optimization、Computer Methods in Applied Mechanics and Engineering、Journal of Computational Mathematics等杂志。
课程题目：The power of gradients in inverse dynamics problems
授课讲者：Tao Du (杜韬), 麻省理工学院（MIT）
课程摘要：Traditionally, inverse dynamics refers to reconstructing forces in a dynamic system from its kinematic motion. This talk considers a broadened definition: inferring various shape, design, and control parameters of a dynamic system from its (partial) observation, which has wide applications in robotics, biomechanics, and machine learning. To solve this problem, we develop a family of computational tools that unleash the power of gradients from physics simulation in many non-traditional ways. We demonstrate these gradient-based methods in parametrizing, modeling, and evaluating rigid-body, deformable-body, and fluidic dynamic systems in simulation and real experiments. We end this talk by discussing open problems in differentiable simulation and its applications in graphics, robotics, and learning.
讲者简介：Tao Du is a Postdoctoral Associate at MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), working with Professor Wojciech Matusik and Professor Daniela Rus. His research aims to combine physics simulation, machine learning, and numerical optimization techniques to solve real-world inverse dynamics problems. His representative works include building differentiable simulation platforms for graphics and robotics research, developing computational design pipelines for real-world robots, and understanding the simulation-to-reality gap of dynamic systems. His work has been published in top-tier graphics, learning, and robotics journals and conferences and has been featured by major technical media outlets. Before continuing at MIT as a Postdoctoral Associate, Tao Du obtained his Ph.D. in Computer Science from MIT in 2021 and his Master's in Computer Science from Stanford in 2015.
课程题目：Neural Representation and Rendering of 3D Real-world Scenes
授课讲者：Lingjie Liu (刘玲洁), Max Planck Institute for Informatics
课程摘要：High-quality reconstruction and photo-realistic rendering of real-world scenes are two important tasks that have a wide range of applications in AR/VR, movie production, games, and robotics. These tasks are challenging because real-world scenes contain complex phenomena, such as occlusions, motions and interactions. In this talk, I will introduce our recent work that integrates deep learning techniques into the traditional graphics pipeline for modeling humans and static scenes from RGB images. We have shown the advantages of these new neural approaches over the classical computer graphics methods. Finally, I will discuss challenges and opportunities in this area for future work.
讲者简介：Lingjie Liu is the incoming Aravind K. Joshi endowed Assistant Professor in the Department of Computer and Information Science at the University of Pennsylvania, where she will be leading the Computer Graphics Lab. Currently, Lingjie Liu is a Lise Meitner Postdoctoral Research Fellow working with Prof. Christian Theobalt in the Visual Computing and AI Department at Max Planck Institute for Informatics. She received her Ph.D. degree at the University of Hong Kong in 2019. Before that, she got her B.Sc. degree in Computer Science at Huazhong University of Science and Technology in 2014. Her research interests include neural scene representations, neural rendering, human performance modeling and capture, and 3D reconstruction.
课程题目：Some Computer Graphics Problems in Building Embodied AI Environments
授课讲者：Hao Su (苏昊), 加州大学圣地亚哥分校(UCSD)
课程摘要：In this course, we will review the topic of representation learning for 3D geometry. 3D representation learning has played the key role in applications ranging from 3D scene capturing, editing, understanding, to end-to-end visuomotor agent learning, with broad applications from metaverse, autonomous driving and robotics, to big science problems such as particle physics and protein structure inference. We will introduce some basics and recent progress in the field.
讲者简介：Hao Su has been in UC San Diego as Assistant Professor of Computer Science and Engineering since July 2017. He is interested in algorithms to sense, understand, and act in the physical world. He served on the program committee of multiple conferences and workshops on computer vision, computer graphics, and machine learning. He has been serving as the Area Chair of top vision conferences (CVPR/ICCV/ECCV) since 2019, the Associate Editor of Transactions on Graphics since 2020, and the Associate Editor of robotics conferences (IROS/ICRA) since 2020.
课程摘要：Using the digital computer to simulate dynamic behavior of elastic objects is a highly desired feature in many scientific and engineering research: in computer animation, it provides realistic effects of soft characters; in surgical simulation, it delivers vivid visual experiences to the trainee; in digital fabrication, it couples geometry design and mechanical analysis. While the basic model has been well established for a while, robustly simulating nonlinear and high-resolution deformable objects is still a challenging problem, especially in a collision-rich environment. In this talk, I will share some of our recent efforts on this classic graphics challenge, and how we manage to improve the quality and the efficiency of the simulation simultaneously. First, I would like to introduce a learning-based model reduction, which is able to capture highly nonlinear material behaviors compactly using a deep network. The network is algorithmically integrated with the simulation pipeline so that the simulation is free of any man-made artifacts. Next, we show a novel numerical solution based on interior point method, embedded in a reduced space, to process collisions and contacts among objects. Lastly, I will present an efficient GPU simulation algorithm using a novel linear solver named aggregated Jacobi. Together, we simulate complicated 3D models with hundred thousand DOFs in real time. All the collisions and contacts will be resolved, regardless of the time step, geometry, and velocity.
讲者简介：Dr. Yin Yang is currently an Associate Professor with the School of Computing at University of Utah. Before joining Utah, the birthplace of Computer Graphics, he was a faculty member at Clemson University and University of New Mexico. He received Ph.D. degree of Computer Science from The University of Texas, Dallas in 2013 (the awardee of David Daniel Fellowship). He was a Research/Teaching Assistant at UT Dallas as well as UT Southwestern Medical Center. His research mainly focuses on real-time physics-based computer graphics, animation and simulation with a strong emphasis on interdisciplinarity. He was a Research Intern in Microsoft Research Asia in 2012. He received NSF CRII (2015) and CAREER (2019) awards. Dr. Yang has published over 70 conference/journal articles in areas of computer graphics, animation, machine learning, computer aided design, and medical imaging. He serves as the TPC member for many international conferences and the reviewer for almost all the top journals/conferences in computer graphics and animation.
授课讲者：Jun-Yan Zhu (朱俊彦), Carnegie Mellon University
课程摘要：Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image. In this talk, we investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. We introduce several algorithms, including paired methods such as pix2pix, pix2pixHD, GauGAN, and unpaired methods such as CycleGAN, UNIT, MUNIT, and CUT. Finally, I will also demonstrate applications in different fields such as vision, graphics, and robotics, as well as usages by developers and visual artists.
讲者简介：Jun-Yan is an Assistant Professor with The Robotics Institute in the School of Computer Science of Carnegie Mellon University. He also holds affiliated faculty appointments in the Computer Science Department and Machine Learning Department. Prior to joining CMU, he was a Research Scientist at Adobe Research and a postdoctoral researcher at MIT CSAIL. He obtained his Ph.D. from UC Berkeley and his B.E. from Tsinghua University. He studies computer vision, computer graphics, computational photography, and machine learning. He is the recipient of the Facebook Fellowship, ACM SIGGRAPH Outstanding Doctoral Dissertation Award, and UC Berkeley EECS David J. Sakrison Memorial Prize for outstanding doctoral research. His co-authored work has received the NVIDIA Pioneer Research Award, SIGGRAPH 2019 Real-time Live! Best of Show Award and Audience Choice Award, and The 100 Greatest Innovations of 2019 by Popular Science.