Object Constraint Language based test case optimization with modified Average Percentage of Fault Detection metric

Kunxiang Jin; Kevin Lano

doi:10.1002/smr.2708

Object Constraint Language based test case optimization with modified Average Percentage of Fault Detection metric

Kunxiang Jin^*, Kevin Lano

^*Corresponding author for this work

King's College London

Research output: Contribution to journal › Article › peer-review

Abstract

Testing is one of the most time-consuming and unpredictable processes within the software development life cycle. As a result, many test case optimization (TCO) techniques have been proposed to make this process more scalable. Object Constraint Language (OCL) was initially introduced as a constraint language to provide additional details to Unified Modeling Language models. However, as OCL continues to evolve, an increasing number of systems are being expressed by this language. Despite this growth, a noticeable research gap exists for the testing of systems whose specifications are expressed in OCL. In our previous work, we verified the effectiveness and efficiency of performing the test case prioritization (TCP) process for these systems. In this study, we extend our previous work by integrating the test case minimization (TCM) process to determine whether TCM can also benefit the testing process under the context of OCL. The evaluation of TCO approaches often relies on well-established metrics such as the average percentage of fault detection (APFD). However, the suitability of APFD for model-based testing (MBT) is not ideal. This paper addresses this limitation by proposing a modification to the APFD metric to enhance its viability for MBT scenarios. We conducted four case studies to evaluate the feasibility of integrating the TCM and TCP processes into our proposed approach. In these studies, we applied the multi-objective optimization algorithm NSGA-II and the genetic algorithm independently to the TCM and TCP processes. The objective was to assess the effectiveness and efficiency of combining TCM and TCP in enhancing the testing phase. Through experimental analysis, the results highlight the benefits of integrating TCM and TCP in the context of OCL-based testing, providing valuable insights for practitioners and researchers aiming to optimize their testing efforts. Specifically, the main contributions of this work include the following: (1) we introduce the integration of the TCM process into the TCO process for systems expressed by OCL. This integration benefits the testing process further by reducing redundant test cases while ensuring sufficient coverage. (2) We comprehensively analyze the limitations associated with the commonly used metric, APFD, and then, a modified version of the APFD metric has been proposed to overcome these weaknesses. (3). We systematically evaluate the effectiveness and efficiency of OCL-based TCO processes on four real-world case studies with different complexities.

Original language	English
Article number	e2708
Journal	Journal of Software: Evolution and Process
Volume	36
Issue number	11
Early online date	4 Jul 2024
DOIs	https://doi.org/10.1002/smr.2708
Publication status	Published - Nov 2024

Keywords

genetic algorithm
modified APFD metric
Object Constraint Language
test case optimization

Access to Document

10.1002/smr.2708

Cite this

@article{d5c0f8d034534a8e9f359c3253e71b36,

title = "Object Constraint Language based test case optimization with modified Average Percentage of Fault Detection metric",

abstract = "Testing is one of the most time-consuming and unpredictable processes within the software development life cycle. As a result, many test case optimization (TCO) techniques have been proposed to make this process more scalable. Object Constraint Language (OCL) was initially introduced as a constraint language to provide additional details to Unified Modeling Language models. However, as OCL continues to evolve, an increasing number of systems are being expressed by this language. Despite this growth, a noticeable research gap exists for the testing of systems whose specifications are expressed in OCL. In our previous work, we verified the effectiveness and efficiency of performing the test case prioritization (TCP) process for these systems. In this study, we extend our previous work by integrating the test case minimization (TCM) process to determine whether TCM can also benefit the testing process under the context of OCL. The evaluation of TCO approaches often relies on well-established metrics such as the average percentage of fault detection (APFD). However, the suitability of APFD for model-based testing (MBT) is not ideal. This paper addresses this limitation by proposing a modification to the APFD metric to enhance its viability for MBT scenarios. We conducted four case studies to evaluate the feasibility of integrating the TCM and TCP processes into our proposed approach. In these studies, we applied the multi-objective optimization algorithm NSGA-II and the genetic algorithm independently to the TCM and TCP processes. The objective was to assess the effectiveness and efficiency of combining TCM and TCP in enhancing the testing phase. Through experimental analysis, the results highlight the benefits of integrating TCM and TCP in the context of OCL-based testing, providing valuable insights for practitioners and researchers aiming to optimize their testing efforts. Specifically, the main contributions of this work include the following: (1) we introduce the integration of the TCM process into the TCO process for systems expressed by OCL. This integration benefits the testing process further by reducing redundant test cases while ensuring sufficient coverage. (2) We comprehensively analyze the limitations associated with the commonly used metric, APFD, and then, a modified version of the APFD metric has been proposed to overcome these weaknesses. (3). We systematically evaluate the effectiveness and efficiency of OCL-based TCO processes on four real-world case studies with different complexities.",

keywords = "genetic algorithm, modified APFD metric, Object Constraint Language, test case optimization",

author = "Kunxiang Jin and Kevin Lano",

note = "Publisher Copyright: {\textcopyright} 2024 The Author(s). Journal of Software: Evolution and Process published by John Wiley & Sons Ltd.",

year = "2024",

month = nov,

doi = "10.1002/smr.2708",

language = "English",

volume = "36",

journal = "Journal of Software: Evolution and Process",

issn = "2047-7473",

publisher = "WILEY-BLACKWELL",

number = "11",

}

TY - JOUR

T1 - Object Constraint Language based test case optimization with modified Average Percentage of Fault Detection metric

AU - Jin, Kunxiang

AU - Lano, Kevin

PY - 2024/11

Y1 - 2024/11

N2 - Testing is one of the most time-consuming and unpredictable processes within the software development life cycle. As a result, many test case optimization (TCO) techniques have been proposed to make this process more scalable. Object Constraint Language (OCL) was initially introduced as a constraint language to provide additional details to Unified Modeling Language models. However, as OCL continues to evolve, an increasing number of systems are being expressed by this language. Despite this growth, a noticeable research gap exists for the testing of systems whose specifications are expressed in OCL. In our previous work, we verified the effectiveness and efficiency of performing the test case prioritization (TCP) process for these systems. In this study, we extend our previous work by integrating the test case minimization (TCM) process to determine whether TCM can also benefit the testing process under the context of OCL. The evaluation of TCO approaches often relies on well-established metrics such as the average percentage of fault detection (APFD). However, the suitability of APFD for model-based testing (MBT) is not ideal. This paper addresses this limitation by proposing a modification to the APFD metric to enhance its viability for MBT scenarios. We conducted four case studies to evaluate the feasibility of integrating the TCM and TCP processes into our proposed approach. In these studies, we applied the multi-objective optimization algorithm NSGA-II and the genetic algorithm independently to the TCM and TCP processes. The objective was to assess the effectiveness and efficiency of combining TCM and TCP in enhancing the testing phase. Through experimental analysis, the results highlight the benefits of integrating TCM and TCP in the context of OCL-based testing, providing valuable insights for practitioners and researchers aiming to optimize their testing efforts. Specifically, the main contributions of this work include the following: (1) we introduce the integration of the TCM process into the TCO process for systems expressed by OCL. This integration benefits the testing process further by reducing redundant test cases while ensuring sufficient coverage. (2) We comprehensively analyze the limitations associated with the commonly used metric, APFD, and then, a modified version of the APFD metric has been proposed to overcome these weaknesses. (3). We systematically evaluate the effectiveness and efficiency of OCL-based TCO processes on four real-world case studies with different complexities.

AB - Testing is one of the most time-consuming and unpredictable processes within the software development life cycle. As a result, many test case optimization (TCO) techniques have been proposed to make this process more scalable. Object Constraint Language (OCL) was initially introduced as a constraint language to provide additional details to Unified Modeling Language models. However, as OCL continues to evolve, an increasing number of systems are being expressed by this language. Despite this growth, a noticeable research gap exists for the testing of systems whose specifications are expressed in OCL. In our previous work, we verified the effectiveness and efficiency of performing the test case prioritization (TCP) process for these systems. In this study, we extend our previous work by integrating the test case minimization (TCM) process to determine whether TCM can also benefit the testing process under the context of OCL. The evaluation of TCO approaches often relies on well-established metrics such as the average percentage of fault detection (APFD). However, the suitability of APFD for model-based testing (MBT) is not ideal. This paper addresses this limitation by proposing a modification to the APFD metric to enhance its viability for MBT scenarios. We conducted four case studies to evaluate the feasibility of integrating the TCM and TCP processes into our proposed approach. In these studies, we applied the multi-objective optimization algorithm NSGA-II and the genetic algorithm independently to the TCM and TCP processes. The objective was to assess the effectiveness and efficiency of combining TCM and TCP in enhancing the testing phase. Through experimental analysis, the results highlight the benefits of integrating TCM and TCP in the context of OCL-based testing, providing valuable insights for practitioners and researchers aiming to optimize their testing efforts. Specifically, the main contributions of this work include the following: (1) we introduce the integration of the TCM process into the TCO process for systems expressed by OCL. This integration benefits the testing process further by reducing redundant test cases while ensuring sufficient coverage. (2) We comprehensively analyze the limitations associated with the commonly used metric, APFD, and then, a modified version of the APFD metric has been proposed to overcome these weaknesses. (3). We systematically evaluate the effectiveness and efficiency of OCL-based TCO processes on four real-world case studies with different complexities.

KW - genetic algorithm

KW - modified APFD metric

KW - Object Constraint Language

KW - test case optimization

UR - http://www.scopus.com/inward/record.url?scp=85197540393&partnerID=8YFLogxK

U2 - 10.1002/smr.2708

DO - 10.1002/smr.2708

M3 - Article

AN - SCOPUS:85197540393

SN - 2047-7473

VL - 36

JO - Journal of Software: Evolution and Process

JF - Journal of Software: Evolution and Process

IS - 11

M1 - e2708

ER -

Object Constraint Language based test case optimization with modified Average Percentage of Fault Detection metric

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this