Customer Churn Prediction Using Pyspark with AI – Driven Insights

“Customer Churn Prediction Using Pyspark with AI - Driven Insights”

V Hema Krishna Anjan 1, S Srujith Reddy 2, C Tarun Kumar 3, Dr Ch Subbalakshmi 4

1,2,3 UG Scholars, 4Professor

1,2,3,4 Department of CSE[Data Science],

1,2,3,4 Guru Nanak Institutions Technical Campus, Hyderabad, Telangana, India

Abstract - This research will forecast customer churn in the telecom industry with the help of PySpark, machine learning algorithms, and AI-powered Insights within the Azure Databricks Platform. The aim is to create a scalable and accurate model that can forecast customers as likely to churn or retain based on past behavior and demographic factors. Random Forest is chosen due to its interpretability and accuracy, and its performance is compared with Logistic Regression, SVM, and XGBoost models on precision, recall, F1-score, and ROC-AUC. Data preprocessing, feature engineering, and exploratory data analysis were carried out with great care. Model performance was measured on AUC, precision, recall, and F1-score metrics, and MLflow was utilized for experiment tracking. Generative AI was also used to interpret model outputs and create actionable business insights. Drivers of churn such as tenure, contract type, and monthly charges were determined, and strategic recommendations were provided for different customer segments. This hybrid approach demonstrates how AI-powered analytics can be utilized for customer retention support in telecom.[1]

Key Words: Customer Churn Prediction, PySpark, Azure Databricks, AI-Driven Insights, Telecom Analytics, Generative AI