Computer Vision: Python Face Swap & Quick Deepfake in Colab

Custom Face Swap using Python and OpenCV & Deepfake Image Animation using 'First Order Motion Model' paper in Colab

What you'll learn

Python based Custom Face Swap Application with Image, Video and Camera. Deepfake Videos based on First Order Motion Model Image Animation Paper

Requirements

A decent configuration computer (preferably Windows) and an enthusiasm to research with Deepfake Technology

Description

Hello and welcome to my new course 'Python Face Swap & Quick Deepfake using Google Colab'

You know, there is is an old phrase that says 'seeing is believing'. But in the world of 'Deepfake' what we see is not always true. Let's define deep fake with the help of an example video. Here is it.


I used only a single stamp size photos of these people to create these videos. Yes, you heard it right. With only a single face image and few minutes of time with a normal computer, you can create deepfake of anyone. You an even make dead people talk or even sing for you.


Yes! Be prepared to be amazed. Before we proceed any further, let me explain the list of contents that are included in this course. This course is divided exactly into two halves.


In the first half, we will be creating a basic python based face swap application. Before we proceed, we will have an introduction to the deep fake technique, its applications, advantages and disadvantages. Then we have to prepare our computer with all the dependencies installed. We will be installing Anaconda, the platform and IDE for our python programming. Later there are few optional sessions for those who want to learn the basics of python programming language basics.


Later we will install the rest of dependences required to built our custom python face swap application. After that we will writing the python code line by line to complete the whole program with more than 300 lines. Alternatively you may also download the complete code from the google drive link provided in the last session of this course. At first we will be doing the face swap using two static images. One as the source image and other as the target image. Later we will try it for realtime video from our computer's web camera. And then we will modify it to make it work with a pre-saved video saved in our computer.


From the examples, you can see that this was just a basic face swap program and is not at all perfect. We were doing it just to learn how things works behind the scenes.


Later we will proceed with implementing deepfake based on a paper called ‘First Order Motion Model for Image Animation’ Submitted to Cornell University by Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci and Nicu Sebe


Since training a deepfake involve expensive GPUs, we have an alternate plan to use the google Colab's free GPU. We will be preparing our google drive by creating folders and uploading the sample driving video, based on which the target image needs to be animated and also the target images or source images.


Also we will be downloading a copy the demo google colab notebook and connect with google drive. Then we will clone the first order motion model repository from google drive.


Later we will also proceed with cloning the face-alignment repository. We will install and set it up in our google colab. Then we will move the files into corresponding folders and start with cropping the driving video using a built in python program.


After that we will download the frozen inference graph of the already trained model to our google drive. And now its all set to proceed with the animation of the source images based on the driving video. Once completed we will download the animated video. We will also do the same for few of the other source images too.


The animated video will not be having audio. So we have to mix the audio to it using any free or opensource video editing tool available online. We will do that in our next session and finally we have all the deepfake animated videos with audio included.


As the final session, we will also discuss how we can save the limited free GPU time offered by google and the workaround if the GPU time is exceeded.


Just another word before I conclude. Please please make use of the contents and techniques mentioned in this tutorial very responsibly. Its intended only for learning and research purpose. Me as an instructor or the platform in which I host this course will not be liable for any illegal or irresponsible use of this technique.


That's all about the topics which are currently included in this quick course. The code, images and weights used in this course has been uploaded and shared in a folder. I will include the link to download them in the last session or the resource section of this course. You are free to use the code in your projects with no questions asked.


Also after completing this course, you will be provided with a course completion certificate which will add value to your portfolio.


So that's all for now, see you soon in the class room. Happy learning and have a great time.


Bibliographies and Reference Credits

NIPS Proceedings - First Order Motion Model for Image Animation - Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe

Cornell University - Computer Vision and Pattern Recognition - First Order Motion Model for Image Animation

Github - AliaksandrSiarohin - first-order-model

Github Pages - First Order Motion Model for Image Animation

Learn OpenCV - Delaunay Triangulation and Voronoi Diagram using OpenCV

Learn OpenCV - Face Swap using OpenCV - Satya Mallick

pysource - Face swapping - Sergio Canu

Who this course is for:

Beginners or who those wants to start with Python based Face Swap and Deepfake Technology