AI-Powered Virtual Cosmetic Try-On and Shade Recommendation for E-Commerce Platform

The AI-Powered Virtual Cosmetic Try-On and Shade Recommendation System for E-Commerce provides an integrated web-based solution for addressing the major challenges associated with purchasing cosmetic products online. The proposed system combines product browsing, AI-based shade recommendation, virtual lipstick try-on, shade comparison, cart management, and online purchasing within a single e-commerce platform. The system uses computer vision and image-processing techniques to detect facial and lip regions, analyse relevant skin characteristics, recommend suitable lipstick shades, and render the selected shade on the user's lips. The virtual try-on component uses facial landmark detection, lip segmentation, colour processing, and blending techniques to generate a realistic preview. The system is directly connected with the e-commerce product-selection and purchase workflow, allowing users to move from shade visualization to cart and checkout. The proposed methodology combines facial landmark detection, skin-tone analysis, machine learning, lip segmentation, OpenCV-based colour processing, and e-commerce integration. The system can be evaluated using parameters such as detection accuracy, processing speed, rendering quality, usability, user satisfaction, and performance under different faces, skin tones, lighting conditions, and camera angles. Overall, the proposed system aims to reduce uncertainty during online cosmetic shopping by allowing customers to visualize lipstick shades before purchasing. It provides a more interactive shopping experience and has the potential to reduce incorrect shade selections and product returns. The system also provides a scalable foundation for future cosmetic applications such as virtual eyeshadow, blush, foundation, and other AI-assisted makeup recommendations.

[1]   T. Li, R. Qian, C. Dong, S. Liu, Q. Yan, W. Zhu, and L. Lin, “BeautyGAN: Instance-Level Facial Makeup Transfer with Deep Generative Adversarial Network,” in Proceedings of the 26th ACM International Conference on Multimedia (MM ’18), Seoul, Republic of Korea, 2018, pp. 645–653. doi: 10.1145/3240508.3240618. Available: https://dl.acm.org/doi/10.1145/3240508.3240618   [2]   H. Chang, J. Lu, F. Yu, and A. Finkelstein, “PairedCycleGAN: Asymmetric Style Transfer for Applying and Removing Makeup,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 40–48. doi: 10.1109/CVPR.2018.00012. Availabe: https://openaccess.thecvf.com/content_cvpr_2018/html/Chang_PairedCycleGAN_Asymmetric_Style_CVPR_2018_paper.html   [3]   S. Saito, T. Li, and H. Li, “Real-Time Facial Segmentation and Performance Capture from RGB Input,” in Computer Vision – ECCV 2016, Lecture Notes in Computer Science, vol. 9914, Springer, 2016, pp. 244–261. doi: 10.1007/978-3-319-46484-8_15. Available: https://link.springer.com/chapter/10.1007/978-3-319-46484-8_15   [4]   I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,” in Advances in Neural Information Processing Systems 27 (NeurIPS 2014), 2014, pp. 2672–2680. Available: https://proceedings.neurips.cc/paper_files/paper/2014/hash/f033ed80deb0234979a61f95710dbe25-Abstract.html   [5]   H.-J. Chen, K.-M. Hui, S.-Y. Wang, L.-W. Tsao, H.-H. Shuai, and W.-H. Cheng, “BeautyGlow: On-Demand Makeup Transfer Framework With Reversible Generative Network,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 10034–10042. doi: 10.1109/CVPR.2019.01028. Available: https://openaccess.thecvf.com/content_CVPR_2019/html/Chen_BeautyGlow_On-Demand_Makeup_Transfer_Framework_With_Reversible_Generative_Network_CVPR_2019_paper.html   [6]   S. An, J. Chen, Z. Zhu, F. Zhou, Y. Yang, Y. Ma, X. Liu, and H. Zhu, “ARCosmetics: A Real-Time Augmented Reality Cosmetics Try-On System,” Frontiers of Computer Science, vol. 17, no. 4, Art. 174706, 2023. doi: 10.1007/s11704-022-2059-8. Available: https://journal.hep.com.cn/fcs/EN/10.1007/s11704-022-2059-8   [7]   C. Guan, S. Wang, and A. W.-C. Liew, “Lip Image Segmentation Based on a Fuzzy Convolutional Neural Network,” IEEE Transactions on Fuzzy Systems, vol. 28, no. 7, pp. 1242–1251, 2020. doi: 10.1109/TFUZZ.2019.2957708. Available: https://ieeexplore.ieee.org/document/8929014   [8]   S. S. Kamble, C. H. Muntean, and A. A. Simiscuka, “A Hybrid HSV and YCrCb OpenCV-Based Skin Tone Recognition Mechanism for Makeup Recommender Systems,” in 2024 International Wireless Communications and Mobile Computing (IWCMC), 2024, pp. 1224–1229. doi: 10.1109/IWCMC61514.2024.10592313. Available: https://ieeexplore.ieee.org/document/10592313   [9]   P. Afshar, J. Yeon, A. Levitskyy, R. Suresh, and A. Banitalebi-Dehkordi, “Improving the Accuracy of Beauty Product Recommendations by Assessing Face Illumination Quality,” 2023. Available: https://www.amazon.science/publications/improving-the-accuracy-of-beauty-product-recommendations-by-assessing-face-illumination-quality