A Comparative Evaluation of Machine Learning Classification Algorithms for Credit Scoring in Financial Institutions with Limited Data Availability

Abstract

Credit scoring is the fundamental analytical process by which financial institutions evaluate a potential borrower's creditworthiness to determine the likelihood of future default. At its most basic level, it serves as a risk-assessment gateway, transforming a candidate’s historical financial behaviors, such as past repayment consistency, debt-to-income ratios, and credit utilization, into a single, actionable numerical value. In the modern financial landscape, this process has become inextricably linked with Machine Learning (ML), as traditional statistical methods often struggle to capture the complex, non-linear relationships hidden within modern high-dimensional data. By leveraging ML, institutions can move beyond rigid, rule-based systems to identify subtle behavioral patterns that define a borrower's risk profile more accurately.

For this purpose, this paper evaluates the performance of six machine learning classification algorithms: Logistic Regression, Random Forest, Naïve Bayes, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and the Decision Tree for credit scoring under data scarcity, addressing the critical challenge of limited labeled data in financial risk assessment. A rigorous methodology is employed, encompassing data preprocessing, exploratory analysis (EDA), and model development, with evaluation based on accuracy, precision, recall, F1-score, and AUC-ROC. To ensure robustness, models are validated via cross-validation, mitigating overfitting risks inherent in small datasets.

In effect, Random Forest excels as the most effective model, achieving 82.50% accuracy alongside high precision (82.38%), recall (83.15%), and F1-score (82.74%), attributable to its ensemble-based resilience to data sparsity and overfitting. The Decision Tree matches this accuracy and surpasses Random Forest in precision (83.69%), offering interpretability but requiring regularization to curb overfitting. KNN delivers moderate performance but suffers from computational inefficiency, while SVM’s balanced metrics are offset by high complexity. Logistic Regression underperforms due to linearity constraints, and Naïve Bayes fails to generalize, hampered by its independence assumption. The study underscores the superiority of ensemble methods like Random Forest for credit scoring with limited data, while dissecting trade-offs between interpretability and predictive power. By systematically benchmarking algorithms under data constraints, this work bridges a critical gap in credit risk modeling, offering actionable insights for real-world applications.