Working in partnership with ETH Zurich, Disney Research has made a significant technical breakthrough by devising a fully automated, neural network-based face-swapping method that can be used for, for example, changing the age of an actor, incorporating a deceased actor into a scene or for when a stunt double is required.
When a face swap is performed, contrast and light conditions are captured so it looks like the replacement actor was actually present on set in the correct lighting conditions.
The new algorithm starts by actually modifying either or both the source and / or target footage and the motion in the source footage is subtly stabilised and smoothed. It then blends the new face onto the original and generates the in-between frames needed for a smooth and realistic result.
The developers concede that the results are not perfect, but believe they are sufficiently photorealistic to convince viewers that little or no jiggery pokery has been performed. If it is picked up by the production community, we could be about to witness face-swapping that is close to bridging ‘the uncanny valley’.
Click here to see the academic paper presented at the 2020 Eurographics Symposium on Rendering (28 June – 3 July).