Advancing Malware Detection in Network Traffic with Self-Paced Class Incremental Learning

Xu, X ORCID: https://orcid.org/0009-0009-3187-6965, Zhang, X, Zhang, Q ORCID: https://orcid.org/0000-0002-2147-4059, Wang, Y ORCID: https://orcid.org/0000-0001-7763-4261, Adebisi, B ORCID: https://orcid.org/0000-0001-9071-9120, Ohtsuki, T ORCID: https://orcid.org/0000-0003-3961-1426, Sari, H and Gui, G ORCID: https://orcid.org/0000-0003-3888-2881 (2024) Advancing Malware Detection in Network Traffic with Self-Paced Class Incremental Learning. IEEE Internet of Things Journal, 11 (12). pp. 21816-21826.

Preview

Accepted Version Available under License In Copyright. Download (3MB) | Preview

Abstract

Ensuring network security, effective malware detection (MD) is of paramount importance. Traditional methods often struggle to accurately learn and process the characteristics of network traffic data, and must balance rapid processing with retaining memory for previously encountered malware categories as new ones emerge. To tackle these challenges, we propose a cutting-edge approach using self-paced class incremental learning (SPCIL). This method harnesses network traffic data for enhanced class incremental learning (CIL). A pivotal technique in deep learning, CIL facilitates the integration of new malware classes while preserving recognition of prior categories. The unique loss function in our SPCIL-driven MD combines sparse pairwise loss with sparse loss, striking an optimal balance between model simplicity and accuracy. Experimental results reveal that SPCIL proficiently identifies both existing and emerging malware classes, adeptly addressing catastrophic forgetting. In comparison to other incremental learning approaches, SPCIL stands out in performance and efficiency. It operates with a minimal model parameter count (8.35 million) and in increments of 2, 4, and 5, achieves impressive accuracy rates of 89.61%, 94.74%, and 97.21% respectively, underscoring its effectiveness and operational efficiency.