Malware Detection System using Machine Learning Techniques

Malware Detection System using Machine Learning Techniques

Deeksha Pal¹, Ashraf Ali2, Vikas Chaudhary3, Gaurav Kumar4
Dr. A.P. Srivastava 5 & Nitin Kumar Sharma6

1,2,3,4UG Student, Department of Computer Science & Engg., NITRA Technical Campus, UP, India

5Asst. Professor and Head, Department of Computer Science & Engg., NITRA Technical Campus, UP, India

⁶Assistant Professor, Department of Computer Science & Engineering, NITRA Technical Campus, UP, India

Abstract - Malware detection plays a critical role in securing endpoints such as workstations, servers, cloud instances, and mobile devices. It involves identifying and mitigating malicious activities triggered by various forms of malware, including viruses, trojans, worms, ransomware, adware, and spyware.

With the rapid proliferation of sophisticated and evasive malware variants, such as polymorphic and metamorphic malware, traditional signature-based and heuristic approaches are no longer sufficient. These conventional methods often fail to detect previously unseen or obfuscated threats, making the adoption of more intelligent and adaptive systems essential. In response to this challenge, this project explores the application of machine learning techniques to enhance malware detection and classification.

Leveraging publicly available malware datasets, various supervised and unsupervised algorithms—such as Decision Trees, Random Forests, Support Vector Machines (SVM), k-Nearest

Neighbors (k-NN), Naïve Bayes, and Artificial Neural Networks (ANN)—will be employed and evaluated. Additionally, feature extraction and dimensionality reduction techniques such as Principal Component Analysis (PCA) will be applied to improve model performance.

The objective is to assess and compare the models using metrics such as accuracy, precision, recall, F1-score, and confusion matrix analysis.

This project aims to identify the most effective machine learning model for real-time, scalable, and accurate malware detection, contributing to improved cybersecurity practices and proactive threat mitigation.

Keywords: Malware detection, machine learning, enterprise security, information gain, random forest, PE analysis, API calls.