A Multi-Stage Deep Learning Pipeline for Missing Person Identification Using Face Recognition and Vector Similarity Search

Hope in Pixels started from a straightforward observation: the tools needed to automate meaningful parts of the missing person identification process already exist, but they have not often been assembled into a system designed for actual operational use. This study describes an attempt to achieve this. The system combines YuNet face detection, SFace embedding extraction, and Milvus vector search with a composite scoring function that considers age, sex, and geographic context alongside facial similarity. The system, integrated within a FastAPI backend and a React frontend, and supported by containerized infrastructure, was designed for deployment rather than mere demonstration. Functional testing verified the accuracy of the classification logic across various intentional mismatch scenarios. Performance benchmarks indicated a total response time per search of approximately 500–600 milliseconds on standard CPU hardware. More significantly, the system transparently identifies itself as a decision-support tool designed to present plausible matches for human evaluation, rather than serving as an infallible oracle that supplants investigative judgment. The POSSIBLE_MATCH tier and the confirm/reject workflow in the frontend are integral components, reflecting the system's recognition that its scores are probabilistic. Consequently, human involvement is essential prior to undertaking any consequential actions.

1. P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," in Proc. IEEE CVPR, 2001, pp. 511–518. 2. K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499–1503, 2016. 3. J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou, "Retina Face: Single-Shot Multi-Level Face Localization in the Wild," in Proc. IEEE/CVF CVPR, 2020, pp. 5203–5212. 4. W. Wu, H. Peng, and S. Yu, "YuNet: A Tiny Millisecond-Level Face Detector," Machine Intelligence Research, vol. 20, no. 5, pp. 656–665, 2023. 5. J. Deng, J. Guo, N. Xue, and S. Zafeiriou, "ArcFace: Additive Angular Margin Loss for Deep Face Recognition," in Proc. IEEE/CVF CVPR, 2019, pp. 4690–4699. 6.  H. Wang et al., "CosFace: Large Margin Cosine Loss for Deep Face Recognition," in Proc. IEEE/CVF CVPR, 2018, pp. 5265–5274. 7.  F. Schroff, D. Kalenichenko, and J. Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering," in Proc. IEEE CVPR, 2015, pp. 815–823. 8. Y. Zhong and W. Deng, "SFace: Sigmoid-Constrained Hypersphere Loss for Robust Face Recognition," IEEE Transactions on Image Processing, vol. 30, pp. 9587–9598, 2021. 9. J. Johnson, M. Douze, and H. Jegou, "Billion-Scale Similarity Search with GPUs," IEEE Transactions on Big Data, vol. 7, no. 3, pp. 535–547, 2021. 10. L. Wang et al., "Milvus: A Purpose-Built Vector Data Management System," in Proc. ACM SIGMOD, 2021, pp. 2614–2627.