Secure Face De-Identification Framework Integrating StyleGAN Latent Manipulation With FPE

We proposed a reversible facial de-identification framework that integrates symmetric-key format-preserving encryption (FPE) into deep neural network-based face-swapping models. This work was motivated by the increasing need to securely manage facial data in digital systems while enabling lawful restoration when required. The method supports identity removal and recovery without compromising data structure or visual fidelity. The proposed framework is applied to two struc-turally different de-identification networks—FaceShifter and SimSwap—demonstrating its versatility and modularity. Notably, the approach maintains non-identity attributes such as pose and expression while ensuring that the identity information can be reconstructed when authorized. Through extensive qualitative and quantitative evaluations, we show that the framework enables high-fidelity face manipulation with minimal perceptual distortion, achieving a favorable trade-off between privacy and utility. Beyond its technical implementation, this work highlights that reversible de-identification can be securely realized without relying on vulnerable password-based schemes. It also contributes to the broader understanding of how differ-ent network architectures affect de-identification robustness under complex visual conditions. The ability to flexibly embed FPE into different models suggests that the method can be extended to other identity-sensitive domains as well. Future research will aim to improve restoration fidelity under highly unconstrained scenarios and expand applicabil-ity to additional real-world datasets and edge deployments where privacy guarantees are essential.

[1]     A. Miotti and A. Wasil, ‘‘Combatting deepfakes: Policies to address national security threats and rights violations,’’ 2024, arXiv:2402.09581. [2]     European Commission. (2018). Eu Data Protection Rules. Accessed: Sep. 12, 2024. [Online]. Available: https://commission.europa.eu/law/law-topic/data-protection/data-protection-euen [3]     J. Song, J. Kim, and J. Nang, ‘‘Face de-identification using convolutional neural network (CNN) models for visual-copy detection,’’ Appl. Sci., vol. 14, no. 5, p. 1771, Feb. 2024. [4]     Y. Wen, B. Liu, L. Song, J. Cao, and R. Xie, ‘‘Differential private identification protection for face images,’’ in Face De-Identification: Safeguarding Identities in the Digital Era. Cham, Switzerland: Springer, 2024, pp. 75–108. [5]     J. Cao, B. Liu, Y. Wen, R. Xie, and L. Song, ‘‘Personalized and invertible face de-identification by disentangled identity information manipulation,’’ in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2021, pp. 3314–3322. [6]     D. Li, W. Wang, K. Zhao, J. Dong, and T. Tan, ‘‘RiDDLE: Reversible and diversified de-identification with latent encryptor,’’ 2023, arXiv:2303.05171. [7]     D. Xiao, J. Xia, M. Li, and M. Zhang, ‘‘Manipulable, reversible and diversified de-identification via face identity disentanglement,’’ Multimedia Tools Appl., vol. 83, no. 31, pp. 75653–75670, Feb. 2024. [8]     Y. Wen, B. Liu, L. Song, J. Cao, and R. Xie, ‘‘Deep motion flow guided reversible face video de-identification,’’ in Face De-identification: Safeguarding Identities in the Digital Era. Cham, Switzerland: Springer, 2024, pp. 147–176. [9]     M. Bellare, T. Ristenpart, P. Rogaway, and T. Stegers, ‘‘Format-preserving encryption,’’ in Proc. Sel. Areas Cryptography: 16th Annu. Int. Workshop. Alberta, ED, Canada: Springer, Jan. 2009, pp. 295–312. [10]     L. Li, J. Bao, H. Yang, D. Chen, and F. Wen, ‘‘FaceShifter: Towards high fidelity and occlusion aware face swapping,’’ 2019, arXiv:1912.13457.