A Comparative Study of Threshold-Based and Machine Learning Models

Fault prediction in software systems is essential for ensuring software quality and reliability. Traditionally, binary classification based on threshold values has been seen as a simpler and more intuitive method than machine learning and statistical techniques. Threshold-based methods allow developers and testers to classify software components as faulty or non-faulty by checking if specific metric values exceed set thresholds. However, it is important to determine if threshold techniques can achieve prediction performance that matches machine learning algorithms. This paper looks into this issue by building threshold-based models using the concordance probability method, Alves Rankings, and the Vales method, and compares their results with those from machine learning and statistical methods like Naive Bayes, Logistic Regression, AdaBoost, Random Forest, and Bagging. The results show the better classification ability of threshold techniques as compared to machine learning based classifiers.