Abstract
Ransomware poses a significant threat by encrypting files or systems demanding a ransom be paid. Early detection is essential to mitigate its impact. This paper presents an Uncertainty-Aware Dynamic Early Stopping (UA-DES) technique for optimizing Deep Belief Networks (DBNs) in ransomware detection. UA-DES leverages Bayesian methods, dropout techniques, and an active learning framework to dynamically adjust the number of epochs during the training of the detection model, preventing overfitting while enhancing model accuracy and reliability. Our solution takes a set of Application Programming Interfaces (APIs), representing ransomware behavior as input we call “UA-DES-DBN”. The method incorporates uncertainty and calibration quality measures, optimizing the training process for better more accurate ransomware detection. Experiments demonstrate the effectiveness of UA-DES-DBN compared to more conventional models. The proposed model improved accuracy from 94% to 98% across various input sizes, surpassing other models. UA-DES-DBN also decreased the false positive rate from 0.18 to 0.10, making it more useful in real-world cybersecurity applications.