H. Kim, P. Garrido , A. Tewari, W. Xu, J. Thies, M. Nießner, P. Pérez, C. Richardt, Michael Zollhöfer, C. Theobalt, Deep Video Portraits, ACM Transactions on Graphics (SIGGRAPH 2018)
We present a novel approach that enables photo-realistic re-animation of portrait videos using only an input video. In contrast to existing approaches that are restricted to manipulations of facial expressions only, we are the first to transfer the full 3D head position, head rotation, face expression, eye gaze, and eye blinking from a source actor to a portrait video of a target actor. The core of our approach is a generative neural network with a novel space-time architecture. The network takes as input synthetic renderings of a parametric face model, based on which it predicts photo-realistic video frames for a given target actor. The realism in this rendering-to-video transfer is achieved by careful adversarial training, and as a result, we can create modified target videos that mimic the behavior of the synthetically-created input. In order to enable source-to-target video re-animation, we render a synthetic target video with the reconstructed head animation parameters from a source video, and feed it into the trained network -- thus taking full control of the target. With the ability to freely recombine source and target parameters, we are able to demonstrate a large variety of video rewrite applications without explicitly modeling hair, body or background. For instance, we can reenact the full head using interactive user-controlled editing, and realize high-fidelity visual dubbing. To demonstrate the high quality of our output, we conduct an extensive series of experiments and evaluations, where for instance a user study shows that our video edits are hard to detect.
1 view
444
101
4 weeks ago 00:00:39 1
Internal Heating | Short Comic Animation #457
4 weeks ago 00:13:27 1
Plasma balls:The Seraphim have arrived, ready for war! UFOs have descended on the Colorado mountains
4 weeks ago 00:03:17 1
Traitors - SBC (Official Music Video)
4 weeks ago 00:03:02 1
Dapa Deep - Wake Up
4 weeks ago 00:34:15 1
🚨Judge Joe Brown BREAKS DOWN DIDDY’s Shocking Indictments!
4 weeks ago 00:03:44 1
ALICE IN WONDERLAND VILLAIN SONG - I Only Paint in Red Now | Song by Lydia the Bard & Tony
4 weeks ago 00:22:41 1
To tears, the story of how my husband taught me to find joy in a hobby
4 weeks ago 00:08:28 1
Wardruna - Lyfjaberg (Healing-mountain) Official music video
4 weeks ago 00:02:07 1
Belinda Rissa - Love in sync, New Video, Top Models Video, Deep House Music
4 weeks ago 00:03:14 1
JamWayne - Deep South (Official Video)
4 weeks ago 00:47:01 1
Top Music Mix 2024 Best Songs & Remixes MissDeep Playlist 6
4 weeks ago 00:05:22 1
Lipstick Tonic - Kerbera (Official Music Video)
4 weeks ago 00:03:18 1
Falling In Reverse - “Ronald“ (feat. Tech N9ne & Alex Terrible)
4 weeks ago 00:03:35 1
Young All For One Defeats Everyone & Steals Hawks Quirk 「Boku No Hero Academia S7 AMV」I Hate You
4 weeks ago 00:31:19 1
Yakuza Gang Boss Shows Me The Dark Side of Japan 🇯🇵
4 weeks ago 00:47:22 1
RUSSIA vs USA || V1 Beatmaking Battle 2024 || Beatsbyjblack & DaTrackAddict vs Moskvin & Nick Brown
4 weeks ago 00:11:29 1
Coast Guard: Videos Of Titan Sub Debris Ocean Floor OceanGate
4 weeks ago 00:03:27 1
Alan Walker - The Spectre
4 weeks ago 03:08:35 1
Deep Rooftop Chillout 🌙 Beautiful Ambient Chillout Music Mix 🎸 Lounge Vibes for Relaxation