SearchGEM5: Towards Reliable gem5 with Search Based Software Testing and Large Language Models

TY - CHAP

T1 - SearchGEM5: Towards Reliable gem5 with Search Based Software Testing and Large Language Models

AU - Dakhama, Aidan

AU - Even-Mendoza, Karine

AU - Langdon, William B.

AU - Menendez, Hector D.

AU - Petke, Justyna

PY - 2023/12/8

Y1 - 2023/12/8

N2 - We introduce a novel automated testing technique that combines LLM and search-based fuzzing. We use ChatGPT to parameterise C programs. We compile the resultant code snippets, and feed compilable ones to SearchGEM5 — our extension to AFL++ fuzzer with customised new mutation operators. We run thus created 4005 binaries through our system under test, gem5, increasing its existing test coverage by more than 1000 lines. We discover 244 instances where gem5 simulation of the binary differs from the binary’s expected behaviour.

AB - We introduce a novel automated testing technique that combines LLM and search-based fuzzing. We use ChatGPT to parameterise C programs. We compile the resultant code snippets, and feed compilable ones to SearchGEM5 — our extension to AFL++ fuzzer with customised new mutation operators. We run thus created 4005 binaries through our system under test, gem5, increasing its existing test coverage by more than 1000 lines. We discover 244 instances where gem5 simulation of the binary differs from the binary’s expected behaviour.

KW - AI

KW - LLM

KW - SBSE

KW - SBFT

KW - genetic improvement of tests

KW - gem5

UR - https://www.springerprofessional.de/en/searchgem5-towards-reliable-gem5-with-search-based-software-test/26487160

UR - https://link.springer.com/chapter/10.1007/978-3-031-48796-5_14

M3 - Conference paper

VL - 15

BT - 15th Symposium on Search Based Software Engineering (SSBSE)

PB - Springer

CY - United States

ER -