COMPARATIVE ANALYSIS OF MULTI-MODAL DEEPFAKE DETECTION TECHNIQUES FOR DIGITAL FORENSICS

A more comprehensive review into the studied deepfake detection algorithms, the capabilities and shortcomings of these various algorithms based on both AI-ones, manual forensics, and additionally some Hybrid multi-modal approaches. The research makes clear that aiming at high detection accuracy, such as CNNs and transformers (85-96%), the AI-driven models also lack particularly overfitting / easy to fool by adversarial deepfakes and computationally expensive. In practice, the model retraining still matters and should be done with as diverse datasets in order to train better generalization models. Also, the dual role of Generative Adversarial Networks (GAN) — as they can be used for improving deepfake realism as well as do help researchers with detection problems exposes the ethical pitfalls of AI-forensic research.   While somewhat effective in their originality, such manual forensic methods by phoneme- viseme mismatch analysis and frame by frame motion inconsistencies/focus, besides metadata- based detection is considered as the best way of detecting the deepfakes where an AI model may be unable. Nine out of 10, Deepfakes do not have even remotely synchronized lips and speech (92%) and around half of them also commit motion artifacts including irregular blinking and face distortions. Looking deeper into metadata analysis, deepfake videos were observed to have 83%digital fingerprint artefacts as constant compression and encoding are disrupted during manipulation. Nonetheless, such forensic techniques are still very tedious to be properly performed by human experts on a timely manner and cannot fit for real-time solution.   Cross-categories detected the best in our study of various detection frameworks for deepfake decoding and it was clear that the hybrid multi-modal approach was the most reliable method for deepfake detection. With AI Classifier, Forensics and Metadata Validation this approach was able to obtain an accuracy of approximately 95% overall with a drastic drop in false positives (8%) and false negatives (6%). Cross-validation of deepfake attributes from hybrid model among speech, facial expression and metadata enabled improved detection reliability especially for challenging fu (of different type) designed to bypass single-modality techniques. Yet, the hurdles like computation costs and real-time are still there to be cleared. Around6.9 minutes per forensic/video, an average processing time might indicate that further optimization is needed for higher efficiency in forensic and media use cases.   While considerable advances have been made in recent studies on deepfake detection, the issue of long-term effectiveness brings up many immediate challenges that need tackling. The constant deepfakes evolution is going to demand dynamic AI technologies that continuously improve with new deepfake generation methods. Speed vs. Accuracy trade-off is another hot spot that will redirect the optimization concern for building lightweight detection systems that can be deployed in real-time.  

1.                Cohen, A., Rimon, I., Aflalo, E., & Permuter, H. H. (2022). A study on data augmentation in voice anti-spoofing. Speech Communication. 2.                Groh, M., Epstein, Z., Firestone, C., & Picard, R. (2021). Deepfake detection by human crowds, machines, and machine-informed crowds. Proceedings of the National Academy of Sciences, 118(1), e2110013119. 3.                Ionescu, B., Buda, T. S., Giataganas, E., & Popescu, A. (2024). Deepfake media generation and detection in the generative AI era. Computers in Industry, 153, 104883 4.                Hu, S., Li, Y., & Lyu, S. (2020). Exposing GAN-Generated Faces Using Inconsistent Corneal Specular Highlights. 5.                Boháček, M., & Farid, H. (2022). Protecting world leaders against deep fakes using facial, gestural, and vocal mannerisms. Proceedings of the National Academy of Sciences of the United States of America. 6.                Masi, I., Killekar, A., Mascarenhas, R. M., Gurudatt, S. P., & Abdalmageed, W. (2020). Two-branch recurrent network for isolating deepfakes in videos. 7.                Cozzolino, D., & Verdoliva, L. (2020). Noiseprint: A CNN-based camera model fingerprint. IEEE Transactions on Information Forensics and Security, 15, 144-159. 8.                Li, Y., & Lyu, S. (2019). Exposing deepfake videos by detecting face warping artifacts. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 46-52. 9.                Agarwal, S., Farid, H., Gu, Y., He, M., Nagano, K., & Li, H. (2019). Protecting world leaders against deep fakes. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 38-45. 10.             Fridrich, J., & Kodovsky, J. (2012). Rich models for steganalysis of digital images. IEEE Transactions on Information Forensics and Security, 7(3), 868-882.