Real-Time Fraud Detection using Apache Kafka, Apache Spark, and PySpark MLlib

In this study we present a robust scalable and real time fraud detection system for credit card transactions.apache kafka is used for data ingestion and Apache Spark Streaming for real-time data processing, and Spark MLlib for implementation of machine learning models. Macos environment is chosen for deployment of the architecture . We use a 5-node Spark cluster and 3 Kafka brokers. Using Our implementation enables immediate detection of fraudulent transactions, which ensures rapid response and decision-making. The paper elaborates on the methodology, architecture, and execution pipeline and discusses the potential of integrating advanced analytics and visualization in future iterations.