Intriguing Properties of Adversarial ML Attacks in the Problem Space

Research output: Contribution to journalConference paperpeer-review

147 Citations (Scopus)
244 Downloads (Pure)


Recent research efforts on adversarial ML have investigated problem-space attacks, focusing on the generation of real evasive objects in domains where, unlike images, there is no clear inverse mapping to the feature space (e.g., software). However, the design, comparison, and real-world implications of problem-space attacks remain under-explored.

This paper makes two major contributions. First, we propose a novel formalization for adversarial ML evasion attacks in the problem-space, which includes the definition of a comprehensive set of constraints on available transformations, preserved semantics, robustness to preprocessing, and plausibility. We shed light on the relationship between feature space and problem space, and we introduce the concept of side-effect features as the byproduct of the inverse feature-mapping problem. This enables us to define and prove necessary and sufficient conditions for the existence of problem-space attacks. We further demonstrate the expressive power of our formalization by using it to describe several attacks from related literature across different domains

Second, building on our formalization, we propose a novel problem-space attack on Android malware that overcomes past limitations. Experiments on a dataset with 170K Android apps from 2017 and 2018 show the practical feasibility of evading a state-of-the-art malware classifier along with its hardened version. Our results demonstrate that “adversarial-malware as a service’ is a realistic threat, as we automatically generate thousands of realistic and inconspicuous adversarial applications at scale, where on average it takes only a few minutes to generate an adversarial app. Yet, out of the 1600+ papers on adversarial ML published in the past six years, roughly 40 focus on malware [15]—and many remain only in the feature space.

Our formalization of problem-space attacks paves the way to more principled research in this domain. We responsibly release the code and dataset of our novel attack to other researchers, to encourage future work on defenses in the problem space
Original languageEnglish
Pages (from-to)1332-1349
Number of pages18
Journal2020 IEEE Symposium on Security and Privacy
Publication statusPublished - 18 May 2020


  • Adversarial machine learning
  • Evasion
  • Input space
  • Malware
  • Problem space
  • Program analysis


Dive into the research topics of 'Intriguing Properties of Adversarial ML Attacks in the Problem Space'. Together they form a unique fingerprint.

Cite this