Is It Overkill? Analyzing Feature-Space Concept Drift in Malware Detectors

TY - CHAP

T1 - Is It Overkill? Analyzing Feature-Space Concept Drift in Malware Detectors

AU - Chen, Zhi

AU - Zhang, Zhenning

AU - Kan, Zeliang

AU - Cortellazzi, Jacopo

AU - Pendlebury, Feargus

AU - Pierazzi, Fabio

AU - Cavallaro, Lorenzo

AU - Wang, Gang

PY - 2023/3/18

Y1 - 2023/3/18

N2 - Concept drift is a major challenge faced by machine learning-based malware detectors when deployed in practice. While existing works have investigated methods to detect concept drift, it is not yet well understood regarding the main causes behind the drift. In this paper, we design experiments to empirically analyze the impact of feature-space drift (new features introduced by new samples) and compare it with data-space drift (data distribution shift over existing features). Surprisingly, we find that data-space drift is the dominating contributor to the model degradation over time while featurespace drift has little to no impact. This is consistently observed over both Android and PE malware detectors, with different feature types and feature engineering methods, across different settings. We further validate this observation with recent online learning based malware detectors that incrementally update the feature space. Our result indicates the possibility of handling concept drift without frequent feature updating, and we further discuss the open questions for future research.

AB - Concept drift is a major challenge faced by machine learning-based malware detectors when deployed in practice. While existing works have investigated methods to detect concept drift, it is not yet well understood regarding the main causes behind the drift. In this paper, we design experiments to empirically analyze the impact of feature-space drift (new features introduced by new samples) and compare it with data-space drift (data distribution shift over existing features). Surprisingly, we find that data-space drift is the dominating contributor to the model degradation over time while featurespace drift has little to no impact. This is consistently observed over both Android and PE malware detectors, with different feature types and feature engineering methods, across different settings. We further validate this observation with recent online learning based malware detectors that incrementally update the feature space. Our result indicates the possibility of handling concept drift without frequent feature updating, and we further discuss the open questions for future research.

M3 - Conference paper

T3 - IEEE Symposium on Security on Privacy Workshops

BT - Deep Learning Security and Privacy Workshop (DLSP)

PB - IEEE

ER -