Fuzzing-Based Differential Testing For Quantum Simulators

Daniel Blackwell*, Justyna Petke, Yazhuo Cao, Avner Bensoussan

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

1 Downloads (Pure)


Quantum programs are hard to develop and test due to their probabilistic nature and the restricted availability of quantum computers. Quantum simulators have thus been introduced to help software developers. There are, however, no formal proofs that these simulators behave in exactly the way that real quantum hardware does, which could lead to errors in their implementation. Here we propose to use a searchbased technique, grammar-based fuzzing, to generate syntactically valid quantum programs, and use differential testing to search for inconsistent behaviour between selected quantum simulators. We tested our approach on three simulators: Braket, Quantastica, and Qiskit. Overall, we generated and ran over 400k testcases, 2,327 of which found new coverage, and 292 of which caused crashes, hangs or divergent behaviour. Our analysis revealed 4 classes of bugs, including a bug in the OpenQASM 3 stdgates.inc standard gates library, affecting all the simulators. All but one of the bugs reported to the developers have been already fixed by them, while the remaining bug has been acknowledged as a true bug.
Original languageEnglish
Title of host publication16th Symposium on Search-Based Software Engineering
Subtitle of host publicationLecture Notes in Computer Science
Publication statusAccepted/In press - 7 May 2024


  • Fuzzing
  • Differential Testing
  • Quantum Simulators


Dive into the research topics of 'Fuzzing-Based Differential Testing For Quantum Simulators'. Together they form a unique fingerprint.

Cite this