Automated and Efficient Test-Generation for Grid-Based Multiagent Systems

Sina Entekhabi, Wojciech Mostowski, Mohammadreza Mousavi

Research output: Contribution to journalArticlepeer-review

30 Downloads (Pure)

Abstract

Automatic generation of random test inputs is an approach that can alleviate the challenges of manual test case design.
However, random test cases may be ineffective in fault detection and increase testing cost, especially in systems where test execution is resource-
and time-consuming. To remedy this, the domain knowledge of test engineers can be exploited to select potentially effective test cases. To this end,
test selection constraints suggested by domain experts can be utilized either for filtering randomly generated test inputs
or for direct generation of inputs using constraint solvers. In this paper, we propose a domain specific
language (DSL) for formalizing locality-based test selection constraints of autonomous agents and discuss the
impact of test selection filters, specified in our DSL, on randomly generated test cases. We study and compare
the performance of filtering and constraint solving approaches in generating selective test cases
for different test scenario parameters and
discuss the role of these parameters in test generation performance.
Through our study, we provide criteria for suitability of the random data filtering approach versus
the constraint solving one under the varying size and complexity of our testing problem.
We formulate the corresponding research questions and answer them by designing and conducting
experiments using QuickCheck for random test data generation with filtering
and Z3 for constraint solving. Our observations and statistical analysis indicate that applying filters can
significantly improve test efficiency of randomly generated test cases. Furthermore, we observe that
test scenario parameters affect the performance of the filtering and constraint solving approaches differently.
In particular, our results indicate that the two approaches have complementary strengths: random generation and filtering works best for large agent numbers and long paths, while its performance degrades in the larger grid sizes and more strict constraints. On the contrary, constraint solving has a robust performance for large grid sizes and strict constraints, while its performance degrades with more agents and long paths.
Original languageEnglish
JournalACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY
Publication statusAccepted/In press - Aug 2023

Fingerprint

Dive into the research topics of 'Automated and Efficient Test-Generation for Grid-Based Multiagent Systems'. Together they form a unique fingerprint.

Cite this