Verbal-to-Visual AI-Powered Forensic Face Reconstruction

The proposed Verbal to Visual AI-Powered Forensic Face Reconstruction system demonstrates the feasibility of utilising Artificial Intelligence for translating verbal descriptions into visual face representations. The system combines Speech Recognition, Natural Language Processing, AI-based image generation, Computer Vision, and face matching to provide an automated approach for forensic face reconstruction and candidate searching from the database. The results show the ability of the system to generate facial representations from verbal descriptions and match them with stored database images to provide ranked matching results. Thus, the proposed system can help forensic investigators in suspect visualization and identification. The system is a decision support tool, not an identification system.

[1]           H. Han, B. F. Klare, K. Bonnen, and A. K. Jain, “Matching Composite Sketches to Face Photos: A Component-Based Approach,” IEEE Transactions on Information Forensics and Security, vol. 8, no. 1, pp. 191-204, 2013, DOI: 10.1109/TIFS.2012.2228856. [2]           T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, and X. He, “AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks,” in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, vol. 2018, pp. 1316–1324, 2018, DOI: 10.1109/CVPR.2018.00143.  [3]            P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 5967-5976, Jul. 2017. DOI: 10.1109/CVPR.2017.632.  [4]            J. Yu, X. Xu, F. Gao, S. Shi, M. Wang, D. Tao, and Q. Huang, “Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs,” IEEE Trans. Cybern., vol. 51, no. 9, pp. 4350–4362, 2021, DOI: 10.1109/TCYB.2020.2972944.  [5]            I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,” Advances in Neural Information Processing Systems (NeurIPS), vol. 27, 2014.  [6]            X. Wang and X. Tang, “Face photo-sketch synthesis and recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 11, pp. 1955–1967, 2009.  [7]            J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired Image-To-Image Translation Using Cycle-Consistent Adversarial Networks,” in 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2223–2232, 2017.  [8]            T. Karras, S. Laine, and T. Aila, “A Style Based Generator Architecture for Generative Adversarial Networks”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4401–4410, 2019.  [9]            R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684–10695, 2022.  [10]       A. Radford, J. W. Kim, et al., “Learning Transferable Visual Models from Natural Language Supervision,” in Proceedings of the 38th International Conference on Machine Learning (ICML), vol. 139, pp. 8748–8763, 2021.