High-quality facial performance capture for realistic computer-generated characters

Derek Bradley

Every year, millions of people are entertained by video games, computer animated films, and stunning visual effects in live-action movies; and each year, these industries are further advanced by research in computer graphics and related fields. Despite advances, one of the hardest challenges remaining is to design realistic computer-generated (CG) characters. Since we interact with real people on a daily basis, audiences are especially trained to read and recognize body language, individual mannerisms and, most importantly, facial expressions. The dynamics of CG facial animation is perhaps the most essential component of a digital character, because subtle changes in the face can have a significant impact on perceived emotion. For decades, this problem has been approached by motion-capture of real actors, however, so far the results lack the required spatial resolution and temporal dynamics to properly drive a CG character with the fidelity of the real actor. To address this drawback, a significant component of my research deals with high-quality facial performance capture, with the goal of reconstructing human faces at an indistinguishable level of realism. To this end, we have developed new methods in both computer graphics and computer vision for high-resolution face capture from image and video data. Our ability to record a performance and recover the intricate shape details and temporal fidelity of the face has already had a significant impact on the entertainment industry. In this talk I will discuss the main challenges in creating realistic facial animation and describe current practices in the video game and film industries. I will then provide an overview of my research in high-resolution facial performance capture, and discuss the future of data-driven CG facial animation.

About the Speaker

I received a Bachelor of Computer Science degree from Carleton University in 2003, a Masters of Computer Science also from Carleton in 2005, and a PhD in Computer Science from the University of British Columbia in 2010. I then began as a Postdoctoral Researcher at Disney Research Zürich and, in 2012, was promoted to Associate Research Scientist. I am interested in various problems in the field of data-driven computer graphics, in particular facial performance capture, data-driven physical simulation, and the 3D reconstruction of complex natural environments. I enjoy publishing in the top venues for computer graphics and computer vision, and my work has led to 3 patents and has been employed on 3 Hollywood feature films.