Hybrid Spatial–Temporal & Frequency Deepfake Detector

This work introduces a hybrid deepfake detection system that combines spatial, temporal, and frequency based analysis within a unified framework. The model is implemented as a multi-branch spatio temporal architecture, integrating EfficientNet for visual feature extraction with Transformer-based temporal modeling and FFT-driven spectral feature fusion.   The system treats video input as a sequence of RGB frames, from which it extracts spatial representations, captures temporal dependencies, and incorporates frequency-domain information into a single classification pipeline. Experiments conducted on a Kaggle deepfake dataset containing 731 videos show that the proposed model achieves a validation accuracy of approximately 92% on a held out set of 147 samples. The model also maintains balanced performance, with F1-scores of 0.92 for both real and fake classes, and a confusion matrix reflecting relatively low misclassification rates. A comparison with baseline models including CNN-only, CNN combined with Transformer, and CNN combined with FFT demonstrates that the hybrid approach provides measurable improvements in performance. This supports the idea that integrating spatial, temporal, and frequency features leads to more reliable deepfake detection. Overall, the project highlights the feasibility of developing an end to end deepfake detection system using modern deep learning techniques, while also providing empirical support for the effectiveness of multi-branch architectures in this domain.

[1] Kietzmann, T., McClure, J., & Goodfellow, I. (2022). Deepfakes: Trick or treat? Artificial intelligence and the future of deception. Annual Review of Vision Science, 8, 1–25. [2] Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020). DeepFakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64, 131–148. [3] Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2019). FaceForensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). [4] Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020). Celeb-DF: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). [5] Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2020). The Deepfake Detection Challenge (DFDC) dataset. arXiv preprint arXiv:2006.07397. [6] Tan, M., & Le, Q. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning (ICML). [7] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS). [8] Zhuang, H., Wang, P., & Zhang, X. (2022). Deepfake detection using frequency-based features and CNNs. Pattern Recognition Letters, 156, 54–60. [9] Tiwarekar, S. (2026). Deep Fake Detection (DFD) Entire Original Dataset. Kaggle Dataset. Available at: https://www.kaggle.com