Deepstar: A Open Source Deepfake Detection Toolkit

Another restored post: Deepfake as a technology has been recently (since June 2016) seen in the wild and has caused concern with a lot of people. A recently released tool – Deepstar is now here to help you detect deepfake videos. Where does this come into picture from a security point of view? According to me, it directly does not. But, if you remember the 2019 Mark Zukerberg video, you would understand how much of a concern it is for a particular brand. This post will detail how does this deepfake detection tool work and what does it do.

What is Deepstar aka deep*?

Deepstar aka deep* is an open source, AI based toolkit in python that helps you detect deepfake videos. It is also extensible enough to be able to facilitate testing of new detection algorithms. As with all AI based tools, it is more effective with curation of data sets useful for detection of deepfakes. As of now, this open source deepfake detection toolkit supports the following deepfake detection models:

1. Mesonet: The mesonet model presents a method to automatically and efficiently detect face tampering in videos, and particularly focuses on two recent techniques used to generate hyper-realistic forged videos: Deepfake and Face2Face. 
2. Mouthnet: This model concentrates AI techniques at the subject’s mouth.

Now how does Deepstar do all of its AI magic? Well, these are the different techniques it makes use of that are implemented as multiple plugins:

1. adjust_color_transform_set_select_extract_plugin.py: This plugin adjusts color for each transform in a transform set.
2. crop_transform_set_select_extract_plugin.py: This plugin crops each transform in a transform set.
3. default_video_select_extract_plugin.py: This plugin extracts frames and thumbnails from a video to a frame set.
4. fade_transform_set_select_merge_plugin.py: This plugin merges transform sets with a fade effect applied.
5. manual_frame_set_select_curate_plugin.py: This plugin serves a manual frame set curation UI.
6. manual_transform_set_select_curate_plugin.py: This plugin serves a manual transform set curation UI.
7. max_blur_transform_set_select_curate_plugin.py: This plugin automatically curates a transform set and rejects transforms that are more blurry than the ‘max-blur’.
8. max_size_transform_set_select_extract_plugin.py: This plugin resizes each transform in a transform set to max-size if its width or height are greater than max-size.
9. mesonet_video_select_detect_plugin.py: This plugin runs the MesoNet classifier over the selected video. Returns True if the video is a deepfale, False if the video is authentic, and None if no analysis could be performed.
10. min_size_transform_set_select_curate_plugin.py: This plugin automatically curates a transform set and rejects transforms with width or height less than ‘min-size’.
11. mouth_transform_set_select_extract_plugin.py: This plugin extracts just the mouth from an image of a face.
12. mtcnn_frame_set_select_extract_plugin.py: This plugin extracts faces from each frame in a frame set.
13. overlay_image_transform_set_select_merge_plugin.py: This plugin merges one transform set with one image. This is to say that one image is overlaid onto every transform in a transform set and at a specified position.
14. overlay_transform_set_select_merge_plugin.py: This plugin merges transform sets overlaying transform set 1 onto transform set 2 at a specified position.
15. pad_transform_set_select_extract_plugin.py: This plugin pads each transform in a transform set.
16. resize_transform_set_select_extract_plugin.py: This plugin resizes each transform in a transform set.
17. slice_transform_set_select_extract_plugin.py: This plugin extracts a slice from a transform set (a subset).
18. transform_set_frame_set_select_extract_plugin.py: This plugin extracts a frame set to a transform set.

Due to it’s modular nature, Deepstar is very easy to be updated and modified to suit your needs. It’s dependencies as of now are flake8, mock, nose, pylint, ansimarkup, flask, flask_restful, imutils, keras, mtcnn, numpy, opencv-contrib-python-headless, pytube, tensorflow, vimeo-dl and Python 3.6.8. Installation is pretty easy and you can be up and running in a few minutes.

Download Deepstar:

Deepstar can be downloaded from its Github repository here. The installation is pretty simple and do not forget to make use of pipenv.