The world’s leading publication for data science, AI, and ML professionals.

Deepfake detection is super hard!

A review and analysis of Facebook Deepfake Detection Challenge.

!!

Photo licensed from stavros - stock.adobe.com
Photo licensed from stavros – stock.adobe.com

Recent advancements in artificial intelligence (AI) and cloud computing technologies have led to rapid development in the sophistication of audio, video, and image manipulation techniques. This synthetic media content is commonly referred to as "deepfakes[1]." AI based tools can manipulate media in increasingly believable ways, for example by creating a copy of a public person’s voice or superimposing one person’s face on another person’s body.

The Book is now available at Amazon – https://www.amazon.com/Deepfakes-aka-Synthetic-Media-Humanity-ebook/dp/B0B846YCNJ/

Legislation, policy, media literacy, and technology must work in tandem for an effective remedy for malicious use of Deepfakes.

Technical countermeasures used to mitigate the impact of deepfakes fall into three categories: media authentication, media provenance, and deepfake detection.

Media Authentication includes solutions that help prove integrity across the media lifecycle by using watermarking, media verification markers, signatures, and chain-of-custody logging. Authentication is the most effective way to prevent the deceptive manipulation of trusted media because it verifies and tracks integrity throughout the content lifecycle or verify it at the distribution endpoint.

Media provenance includes solutions that provide information on media origin, either in the media itself or as metadata of the media. Reverse media search can also be an effective provenance tool, a list of websites where a specific media appeared in the past can be used to prove the origin of the media. Provenance, together with authentication, can provide important forensic tools to help debunk deepfakes.

Deepfake detection includes solutions that leverage multi-modal detection techniques to determine whether target media has been manipulated or synthetically generated. Existing detection techniques can be loosely split into manual and algorithmic methods. Manual techniques include human media forensic practitioners, often armed with software tools. Algorithmic detection uses an AI-based algorithm to identify manipulated media.

Since most of the deepfakes are created via adversarial training (GANs), the creator algorithm’s ability to evade AI-based detection methods will improve as they are introduced to new detection systems. Any detector will have a short shelf life if the creator will have access to it.

There is a broad consensus in the industry that deepfake detection may solve the problem in the short team but in longer term the practical solution will be authentication and provenance techniques, along with media literacy.

[I will write in more detail on the technical countermeasure in a future article.]

Facebook Deepfake Detection Challenge (Dfdc) Results

The best models got 65% accuracy on real-world data. The results reinforce the difficulty of deepfake detection and emphasize the limitations of AI models to mitigate the synthetic media threat.

In September 2019, Facebook, in partnership with Microsoft, AWS, Partnership on AI announced the Deepfake Detection challenge (DFDC) on Kaggle to invite researchers to develop deepfake detection algorithms. To provide data to train potential algorithms, Facebook hired 3,500 actors to record thousands of videos, which were then manipulated using various deepfake cheapfake creation techniques. A total of 100,000 video clips of original and manipulated content were released for researchers as a training set to build deepfake detector algorithms. Notably, this dataset included a more diverse and inclusive set of actors than previous training dataset available.

Submissions to the challenge were closed on March 31st, and Facebook announced the results of the competition on June 12th.

More than 2000 participants submitted 35,000+ detection algorithms. The top-performing models used Google’s EfficientNet algorithm, a high-performing and scalable convolutional neural network (CNN).

Facebook published a paper detailing the competition here.

The winning teams were able to get 82.56% accuracy on the DFDC training data. While this seems high, the same algorithms got 65.18% precision with unforeseen, real-world deepfakes.

Key Observations from the DFDC results data:

  1. The precision of ~65% means that 35% of deepfakes that the system identified as positives (deepfakes) were false positives (not deepfakes). 1/3rd of deepfakes, classified by the algorithms as deepfakes, were not deepfakes.
  2. Recall, or the measure of how many deepfakes the algorithm identifies as true positives compared to the number of actual deepfakes in the dataset, is around 50% at 65% precision algorithm (see the precision-recall chart below). That is, the algorithm identified a real video as a deepfake (false positive) 50% of the time. Half the time the algorithm was not able to identify a deepfake.
  3. The AUC (Area under the curve) on a ROC (Receiver Operative Characteristic), a graph of true positive rate (TPR) and false positive rate (FPR), is ~0.7. The winning algorithms are not performing particularly well – by comparison, a naïve model, such as a coin tossing algorithm, can achieve an AUC of 0.5[2].

The DFDC results reinforces the difficulty of deepfake detection. The most promising technical countermeasures remain authentication and provenance technology, in addition to media literacy measures.

Partnership on AI (PAI) Recommendation

PAI created the AI and Media Integrity Steering Committee in late 2019 as a formal body of Partners for developing and advising projects that strengthen mis/disinformation solutions, including detection of manipulated and synthetic content. DFDC was the first project of the AI and Media Integrity steering committee. They published a report on key learnings from the DFDC.

Key recommendations on Deepfake detection:

  1. Detection alone will not solve information integrity challenges.
  2. Detection should incorporate real-world examples and thread to be meaningful.
  3. The ability to detect must extend to journalists, fact-checkers and civil society groups in an easy to understand interface.
  4. Deepfake detection solution development faces a tradeoff between open source datasets and models and deterring adversaries who could use those resources to improve deepfakes.
  5. We need a meaningful multi-stakeholder collaborative approach in this issue area.

The report is a great resource to learn the challenges of deepfake detection and recommendations for effective countermeasures.

Technical Analysis of the DFDC results

Glossary of some technical terms and a quick guide of machine learning metric.

True Positive (TP): Deepfakes detected as deepfakes

True Negative (TN): Non-Deepfakes detected as non-deepfakes.

False Positive (FP): Non-Deepfakes detected as deepfakes.

False Negative (FN): Deepfakes detected as non-deepfakes.

Accuracy: is defined as the ratio of all the accurate data found (TP+TN) to all the data (TP+FP+TN+FN). For DFDC context, because of the class skew, accuracy is not used instead, they used weighted precision.

DFDC’s Precision Measure: Since deepfake detection is more critical (classifying true positives) than true negative (not deepfake) and because of the class skew, imbalance of fake vs real videos in real world, the false positive numbers (not deepfakes) will give a high accuracy for any model. Facebook created a weighted precision metric to measure the efficacy of the algorithms. A weight was added to false positives to normalize the false positives to have a practical utility of the detection model.

Precision: also known as specificity, is calculated by true positives deepfakes (TP) out of all the positive deepfakes classified by the algorithm, including false positives, (TP+FP).

Precision = TP / (TP + FP)

Since the precision for DFDC model is 65%, it means that 35% of deepfakes that the system identified as positives (deepfakes) were not deepfakes. The algorithms are misclassifying non-deepfakes as deepfakes 35% of times, which is very high.

Recall: also known as sensitivity of the model, is calculated as the ratio of true positives (deepfakes) and all the real deepfakes in the dataset (TP + FN).

Recall = TP / (TP + FN)

Based on the chart below for .65 precision, the recall is around 50%, which means that the best model categorized a real video as deepfake 50% of times.

F1 Score: Ideally, we want an algorithm that classifies no false positives (precision = 1) and no false negative (recall = 1). So, the performance of an algorithm is typically measured by the F1 Score, which is a harmonic mean of precision and recall. Based on the domain, you can adjust the threshold of acceptable false positives and false negative. Like in terminal disease classification scenario, a false negative may have dire consequences, so a higher recall (low false negative) is desirable. For junk email classifiers, false positive (missing an important email) may not be acceptable but false negative (getting junk) is fine, so they can agree to a low recall but a higher precision.

F1 Score = 2 (Precision Recall) / (Precision + Recall)

For DFDC algorithms the F1 Score (.65 precision and .5 recall) is ~.56, which in my view is not a good performance measure for the DFDC algorithms. the algorithm is erroring on both false positives and false negatives.

Receiver operator characteristic (ROC) curve: is a curve of true positive rate and false positive rate. The Area Under the curve (AUC) is a good indication of the performance of the algorithm. I have highlighted the AUC.

Precision Recall Curve https://arxiv.org/abs/2006.07397
Precision Recall Curve https://arxiv.org/abs/2006.07397
ROC (TPR FPR Curve) https://arxiv.org/abs/2006.07397
ROC (TPR FPR Curve) https://arxiv.org/abs/2006.07397

Liked? Feel free to buy me a book


References:

[1] https://www.vice.com/en_us/article/bjye8a/reddit-fake-porn-app-daisy-ridley

[2] https://people.inf.elte.hu/kiss/11dwhdm/roc.pdf


Related Articles