By: C S Nakul Kalyan, CCRI, Asia University, Taiwan

Abstract

With the advancements of Facial Manipulation Techniques, expression-based Forgeries such as Deep Fakes has become a big challenge for the Multimedia Forensics. These manipulations has been done by altering the Facial part ex- cept the features like hair,ears and neck untouched.Here we propose a Deepfake Detection Framework which combines the attention based multitasking with a dual network architecture which can detect both the facial and contextual discrepancies.This method utilizes the both Face identification and the content recognition network, where the outputs have been got from the attention-guided Decoder. we evaluate on a large scale benchmark which contains over 1.8 million manipulated data and the datasets which we are using are FaceForencies++,caleb- DF-V2 and DFDC which demonstrates that our method achieves good perfor- mance in both Training and Testing phases and can help to reduce the Forgeries which are created by unseen Techniques

Keywords

Deepfake, Face-forgery, Face swapping, Deep learning

Introduction

The advancement of the generative models has led to realistic Face manipu- lations which is known as Deep Fakes.The Techniques of Face swapping and expression reenactment can alter a person’s identity or it can change and re- place the expressions of them in images and videos as shown in (Figure 1). while this advancements has many advantages,it also has a serious concerns about the misinformation,Privacy and digital trust,especially when the manipulated data spreads in the Social platforms. Traditional Deepfake detections approaches rely on deep neural network which performs binary classifications, which are struggling with the unseen manipulated data or methods.Recent studies have

highlighted that Face swapping alters only the internal face while leaving the hair ears, neck untouched [2].The attention based techniques will make the models to focus on the manipulated regions which improves the performance in both within-dataset and cross -dataset scenarios [5].

Use Cases and Related Work

the usage of face swapping has initially introduced for the purpose of preserving privacy and providing environment, now the face swapping has been evolved since past 2 decades. The early face swapping methods relied on 3D Morphable models(3DMM) which are used for expression transfer and 3D based manipulation tools.

With the usage of Deep Learning, especially Generative adversarial networks (GANs), the facial manipulations have reached the unpredicted level of realism where the high quality fake images and videos were produced [1]

For these threats rising, the early deep fake detection using the deep learning approaches such as the convolutional neural networks(CNNs), Recurrent neural networks (RNNs),attention mechanism were introduced to detect the forgeries [5] [4].

several Datasets which includes Face Forensics, Face Forensics++ [3],Celeb

– DF, DFDC (Deepfake Detection Challenge) and others which consists of the varius manipulated media which is generated by Different methods and levels where it is used to increase the accuracy level of the model’s detection Algorithm as mentioned in (Table 1).

The methods which are used to detect based on the differences between the faces and their context has shown good results in identifying the deepfake content [2].

Table 1: Comparison of Deepfake detection Datasets

DATASET	SIZE	MANIPULATION METHODS	YEAR
FaceForensics++	1.8 M + Videos	Face2Face,Deepfake,FaceSwap	2019
Celeb-DF-V2	5639 Videos	Advanced synthesis methods	2020
DFDC	128,154 Videos	Unknown methods	2020

The multi-attention-based methods have shown improvement For the identification and localization of deepfake faces and expression swaps [5-10].

Conclusion

The increasing threat and realism of the Deepfake technologies is a major societal and the security challenge. while the visual quality of the deepfake improving it make the detection more difficult. This paper states that the identifying the deepfake face manipulated content has been developing .with the combined insights of the attention mechanisms and dual-network strategy which exploits discrepancies between the facial and contextual regions demonstrates generalization capabilities across the unseen datasets and acheive the major accurate benchmarks. These strategies focuses on the inconsistencies which is produced from the manipulation methods.

References

1. Wasin Al Kishri, Jabar H Yousif, Mahmood Al Bahri, Muhammad Zakarya, Naveed Khan, Sanad Sulaiman Al Maskari, and Ahmet Gurhanli. A com- parative study of deepfake facial manipulation technique using generative adversarial networks. Discover Artificial Intelligence, 5(1):109, 2025.
2. Yuval Nirkin, Lior Wolf, Yosi Keller, and Tal Hassner. Deepfake detection based on discrepancies between faces and their context. IEEE transactions on pattern analysis and machine intelligence, 44(10):6111–6121, 2021.
3. Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manip- ulated facial images. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 1–11, 2019.
4. ST Suganthi, Mohamed Uvaze Ahamed Ayoobkhan, Nebojsa Bacanin, K Venkatachalam, Hub´alovsky` Sˇtˇep´an, Trojovsky` Pavel, et al. Deep learn- ing model for deep fake face recognition and detection. PeerJ Computer Science, 8:e881, 2022.
5. Saima Waseem, Syed Abdul Rahman Syed Abu-Bakar, Zaid Omar, Bi- lal Ashfaq Ahmed, Saba Baloch, and Adel Hafeezallah. Multi-attention-based approach for deepfake face and expression swap detection and lo- calization. EURASIP Journal on Image and Video Processing, 2023(1):14, 2023.
6. Al-Ayyoub, M., AlZu’bi, S., Jararweh, Y., Shehab, M. A., & Gupta, B. B. (2018). Accelerating 3D medical volume segmentation using GPUs. Multimedia Tools and Applications, 77(4), 4939-4958.
7. Gupta, S., & Gupta, B. B. (2015, May). PHP-sensor: a prototype method to discover workflow violation and XSS vulnerabilities in PHP web applications. In Proceedings of the 12th ACM international conference on computing frontiers (pp. 1-8).
8. Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., & Nayar, S. K. (2008). Face swapping: automatically replacing faces in photographs. In ACM SIGGRAPH 2008 papers (pp. 1-8).
9. Zhu, Y., Li, Q., Wang, J., Xu, C. Z., & Sun, Z. (2021). One shot face swapping on megapixels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4834-4844).
10. Zhang, Y., Zheng, L., & Thing, V. L. (2017, August). Automated face swapping and its detection. In 2017 IEEE 2nd international conference on signal and image processing (ICSIP) (pp. 15-19). IEEE.

Cite As

Kalyan C.S.N. (2025) Face Swapping in Videos: Unmasking the Digital Doppelganger, Insights2Techinfo, pp.1

866010cookie-checkFace Swapping in Videos: Unmasking the Digital Doppelgangeryes