Clustering Test Cases to Achieve Effective & Scalable Prioritisation Incorporating Expert Knowledge

TY - CHAP

T1 - Clustering Test Cases to Achieve Effective & Scalable Prioritisation Incorporating Expert Knowledge

AU - Yoo, Shin

AU - Harman, Mark

AU - Tonella, Paolo

AU - Susi, Angelo

PY - 2009

Y1 - 2009

N2 - Pair-wise comparison has been successfully utilised in order to prioritise test cases by exploiting the rich, valuable and unique knowledge of the tester. However, the prohibitively large cost of the pairwise comparison method prevents it from being applied to large test suites. In this paper, we introduce a cluster-based test case prioritisation technique. By clustering test cases, based on their dynamic runtime behaviour, we can reduce the required number of pair-wise comparisons significantly. The approach is evaluated on seven test suites ranging in size from 154 to 1,061 test cases. We present an empirical study that shows that the resulting prioritisation is more effective than existing coverage-based prioritisation techniques in terms of rate of fault detection. Perhaps surprisingly, the paper also demonstrates that clustering (even without human input) can outperform unclustered coverage-based technologies, and discusses an automated process that can be used to determine whether the application of the proposed approach would yield improvement.

AB - Pair-wise comparison has been successfully utilised in order to prioritise test cases by exploiting the rich, valuable and unique knowledge of the tester. However, the prohibitively large cost of the pairwise comparison method prevents it from being applied to large test suites. In this paper, we introduce a cluster-based test case prioritisation technique. By clustering test cases, based on their dynamic runtime behaviour, we can reduce the required number of pair-wise comparisons significantly. The approach is evaluated on seven test suites ranging in size from 154 to 1,061 test cases. We present an empirical study that shows that the resulting prioritisation is more effective than existing coverage-based prioritisation techniques in terms of rate of fault detection. Perhaps surprisingly, the paper also demonstrates that clustering (even without human input) can outperform unclustered coverage-based technologies, and discusses an automated process that can be used to determine whether the application of the proposed approach would yield improvement.

M3 - Conference paper

SN - 978-1-60558-338-9

T3 - ISSTA 2009: INTERNATIONAL SYMPOSIUM ON SOFTWARE TESTING AND ANALYSIS

SP - 201

EP - 211

BT - Unknown

PB - ASSOC COMPUTING MACHINERY

CY - NEW YORK

T2 - ISSTA 2009 - Intternational Symposium on Software Testing and Analysis

Y2 - 19 July 2009 through 23 July 2009

ER -